DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. Claims 1-20 are pending.
Information Disclosure Statement
3. The information disclosure statements (IDSs) submitted on 1/2/2025 and 8/14/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Drawings
4. The drawings have been reviewed and are accepted as being in compliance with the provisions of 37 CFR 1.121.
Claim Objections
5. Claims 1-20 are objected to because of the following informalities: Claims 1-2, 9-10, 16-17 recite “multi-modal predication”. The specification recites “multi-modal prediction”. Appropriate correction is required.
Claim Rejections - 35 USC § 103
6. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
7. Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al (US 2018/0032846), in view of FRANK et al (US FRANK et al (CN 111954564, filed on January 21, 2018), hereinafter “Yang” and “FRANK”.
As per Claim 1, Yang discloses:
A method for generating a trained prediction model, the method comprising: receiving, by one or more processors, event data for a plurality of events from a data store, wherein the event data includes spatial event data and non-spatial event data; (Par [0003], “…multimedia event detection, semantic indexing, gesture control, etc.” and par [0028], “…such as a 2D or 3D CNN during step 110, classification accuracy data is computed for spatial regions associated with each image across a video (i.e., frames or snippets in a video sequence).” And par [0029], “…Each feature map is divided into non-overlapping spatial regions including one or more features or elements x.sub.i that each correspond to a portion of the pixels (e.g., 2×2 pixel regions)...” The event data being the “retrieval of videos” with the multimedia event detection, as claimed see also Figures 1A-1D) transforming, by the one or more processors, the event data into one or more segmented data sets; (Par [0003], “Content based video classification is fundamental to intelligent video analytics (IVA) and includes automatic categorizing, searching, indexing, segmentation, and retrieval of videos… multimedia event detection...” The event data being the “retrieval of videos” with the multimedia event detection, as claimed see also Figures 1A-1C and par [0059], “the first layer of the CNN to extract sets of feature maps for each CNN layer slice 138” and figure 1D) creating, by the one or more processors, an embedding vector of sparse data included in the one or more segmented data sets; (Par [0023], “A video image data classification technique extracts features by processing input image data or snippet data (i.e., a short video clip) using a set of learned parameters to generate a class label for the video (i.e., classification output data). Each layer of a convolutional neural network (CNN) extracts features.” And see Figures 1D and 1F, par [0055], “…l feature descriptors, each of which includes d.sub.l elements, f.sub.t,l is far more sparse as spatial information is lost. Considering these properties, temporal max pooling may be applied to aggregate f.sub.t,l across time to obtain f.sub.l, which is the initial representation of video data at the l-th FC layer.”) generating, by the one or more processors, one or more input data sets based on the embedding vector; (Par [0038-0039], “Input video image data may be presented in the form of single frames C, to the 2D-CNNs 125 for extracting local spatial-temporal features by applying learnable filters at each time t to generate a set of feature maps for each 2D-CNN layer. Each learnable filter is a multi-dimensional kernel that is determined when the 2D-CNN 125 is trained.” And ) and training, by the one or more processors, a mixture density network to generate a multi-modal predication based on the one or more input data sets. (Par [0036], “…a powerful fusing model may be employed to learn the optimal combination of multilayer and multimodal predictions to produce a class label.” Par [0046-0048], “The features x.sub.i have a Gaussian mixture model (GMM) distribution characterized by parameters” And paragraphs [0061-0062], “A coefficient may be learned during training for each CNN layer and each prediction for the layer may be scaled by the coefficient and summed to generate a class label by multilayer fusing. Similarly, a coefficient may be learned for each modality, and each prediction by a 2D-CNN or a 3D-CNN may be scaled by the coefficient for the modality and summed to generate a class label by multimodal fusing.“).
Yang discloses different vector characterizations and calculations, however not specifically the “embedding” as claimed.
FRANK discloses the above claimed features as follows: (Page 13, par 7, “In step 414, the game module 202 can learn how to be based on team history embedding, player history embedding, latest team member embedding and game background to predict the result of the game. For example, the team history is embedded, player embedding, the combination of the nearest player embedding and game background as input, to train the neural network 310. The weight of the neural network 310 may be calculated, for example, by reducing (e.g., minimizing the average cross entropy loss on the data set).”).
Therefore, it would have been obvious to a person of ordinary skill in the art at the effective filing date to incorporate the teachings of FRANK specifically an embedding vector related to a specific event into the method of Yang to take advantage on applying certain event data with team specific information for a specific learning embedded feature. The modification would have been obvious because one of the ordinary skills in the art would implement providing team member embedding specific information to provide the best prediction possible.
As per Claim 2, the rejection of Claim 1 is incorporated and Yang further discloses the method further comprising: reducing, by the one or more processors, a loss between a predicted value of the multi-modal predication and an actual value. (Par [0044], “Each convlet is a one-dimensional vector of values corresponding to a set of feature maps aggregated in spatial and temporal dimensions and the convlets are stored in a data structure. The convlets measure the spatial discriminability of activations at a CNN layer…” And par [0046-0048], “…and reduce feature dimensions. Each feature x.sub.i is then encoded by the deviations with respect to the parameters of GMM. Let γ.sub.i,k be the soft assignment of x.sub.i to the k-th Gaussian component:…”and Par [0054], “The outputs of the FCs 135 are processed by a temporal max pooling layer 132 that applies a down-sampling operator to reduce temporal dimensions of internal multi-dimensional tensor and generate a prediction.”)
Yang, discloses a down sampling, do not specifically disclose the loss.
However, FRANK discloses the above claimed features as follows: (Page 13, par 7, “In step 414, the game module 202 can learn how to be based on team history embedding, player history embedding, latest team member embedding and game background to predict the result of the game. For example, the team history is embedded, player embedding, the combination of the nearest player embedding and game background as input, to train the neural network 310. The weight of the neural network 310 may be calculated, for example, by reducing (e.g., minimizing the average cross entropy loss on the data set).”).
Therefore, it would have been obvious to a person of ordinary skill in the art at the effective filing date to incorporate the teachings of FRANK specifically an embedding vector related to a specific event into the method of Yang to take advantage on applying certain event data with team specific information for a specific learning embedded feature. The modification would have been obvious because one of the ordinary skills in the art would implement providing team member embedding specific information to provide the best prediction possible.
As per Claim 3, the rejection of Claim 1 is incorporated and Yang further discloses:
the method further comprising: parsing, by the one or more processors, the event data to generate one or more sets of data corresponding to each event in a match. (Par [0041], “Therefore, sets of features for appropriate levels of compositionality for the different CNN layers supply diverse fine-scale information for classification. Meanwhile, the feature maps from each layer are already generated for input to the subsequent layer, so that no additional computation is performed to produce the sets of feature maps.” Generates sets of feature maps)
However, Yang do not specifically disclose that the data corresponds to each event in a match.
FRANK discloses the above claimed features as follows (Page 2, “ The computing system selects one or more features related to the current background of the sporting event from the event data…”).
Therefore, it would have been obvious to a person of ordinary skill in the art at the effective filing date to incorporate the teachings of FRANK specifically an embedding vector related to a specific event into the method of Yang to take advantage on applying certain event data with team specific information for a specific learning embedded feature. The modification would have been obvious because one of the ordinary skills in the art would implement providing team member embedding specific information to provide the best prediction possible.
As per Claim 4, the rejection of Claim 1 is incorporated and FRANK further discloses: wherein the one or more segmented data sets include a playing surface position, a play-by-play event sequence, one or more players, one or more teams, a possession team, or a game context. (Page 5, par 5, “can be the tracking system 102 configured for each frame in the game file 110, at least storing the player identity and position information (e.g., (x, y) position) of all players and objects on the game field…”).
Therefore, it would have been obvious to a person of ordinary skill in the art at the effective filing date to incorporate the teachings of FRANK specifically an embedding vector related to a specific event into the method of Yang to take advantage on applying certain event data with team specific information for a specific learning embedded feature. The modification would have been obvious because one of the ordinary skills in the art would implement providing team member embedding specific information to provide the best prediction possible.
As per Claim 5, the rejection of Claim 1 is incorporated and Yang further discloses: the method further comprising: providing. by the one or more processors, the sparse data to one or more embedding layers, wherein the one or more embedding layers are configured to output the embedding vector of the sparse data. (Par [0055], “…l feature descriptors, each of which includes d.sub.l elements, f.sub.t,l is far more sparse as spatial information is lost. Considering these properties, temporal max pooling may be applied to aggregate f.sub.t,l across time to obtain f.sub.l, which is the initial representation of video data at the l-th FC layer.”).
Yang do not specifically disclose the “embedding”.
FRANK discloses the above claimed features as follows: (Page 13, par 7, “In step 414, the game module 202 can learn how to be based on team history embedding, player history embedding, latest team member embedding and game background to predict the result of the game. For example, the team history is embedded, player embedding, the combination of the nearest player embedding and game background as input, to train the neural network 310. The weight of the neural network 310 may be calculated, for example, by reducing (e.g., minimizing the average cross entropy loss on the data set).”).
Therefore, it would have been obvious to a person of ordinary skill in the art at the effective filing date to incorporate the teachings of FRANK specifically an embedding vector related to a specific event into the method of Yang to take advantage on applying certain event data with team specific information for a specific learning embedded feature. The modification would have been obvious because one of the ordinary skills in the art would implement providing team member embedding specific information to provide the best prediction possible.
As per Claim 6, the rejection of Claim 1 is incorporated and Yang further discloses: wherein the generating the one or more input data sets based on the embedding vector includes: concatenating, by the one or more processors, the embedding vector with one or more continuous features. (Paragraphs [0048-0049], “The modified aggregated feature descriptor (i.e., wFV) representation of video input data at a CNN layer may be obtained by concatenating the following derivative vectors from K Gaussian components:…” and see figures 1-2C).
Yang do not specifically disclose the “embedding”.
FRANK discloses the above claimed features as follows: (Page 13, par 7, “In step 414, the game module 202 can learn how to be based on team history embedding, player history embedding, latest team member embedding and game background to predict the result of the game. For example, the team history is embedded, player embedding, the combination of the nearest player embedding and game background as input, to train the neural network 310. The weight of the neural network 310 may be calculated, for example, by reducing (e.g., minimizing the average cross entropy loss on the data set).”).
Therefore, it would have been obvious to a person of ordinary skill in the art at the effective filing date to incorporate the teachings of FRANK specifically an embedding vector related to a specific event into the method of Yang to take advantage on applying certain event data with team specific information for a specific learning embedded feature. The modification would have been obvious because one of the ordinary skills in the art would implement providing team member embedding specific information to provide the best prediction possible.
As per Claim 7, the rejection of Claim 1 is incorporated and FRANK further discloses: wherein the one or more continuous features include a score difference, a remaining time, or a playing surface position. (Page 16, par 8-9, “output from the mixed density network 224 can be N = 10 Gaussian distribution of the mixed parameter, so that the game end score difference can have the following distribution…”).
Therefore, it would have been obvious to a person of ordinary skill in the art at the effective filing date to incorporate the teachings of FRANK specifically an embedding vector related to a specific event into the method of Yang to take advantage on applying certain event data with team specific information for a specific learning embedded feature. The modification would have been obvious because one of the ordinary skills in the art would implement providing team member embedding specific information to provide the best prediction possible.
As per Claim 8, the rejection of Claim 1 is incorporated and FRANK further discloses: the method further comprising: outputting, by the one or more processors, a game state vector based on the one or more input data sets. (Page 2, par 5, the current state of each event at time t, the current score record table of each time t and the data set of one or more players in each time t may appear in each event, learning the score difference of each time t. The computing system receives an indication that a predicted outcome of the sporting event is generated” and page 16, “The game module 204 may generate the first vector by parsing the one or more information sets received from the data storage unit 118, and identifies those information sets for the team metric. For example, the game module 204 can construct the first vector Xt of the score record table of each team…”).
As per Claims 9-20, being the non-transitory computer readable medium and system method claims corresponding to the system claims 1-8 respectively and rejected under the same reason set forth in connection of the rejections of Claims 1-8 and further Yang discloses: (Par [0049]).
Conclusion
8. The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Carr; George Peter Kenneth, US-20180157974-A1, relates to In an embodiment, “ghosted” player movements may refer to a prediction of the actions and/or movements one or more players may have conducted if those players acted in accordance with a particular predetermined ghosting model.
Gamon; Michael, US-20150213361-A1, relates to the transition modeling module 130 uses machine learning techniques to generate the aforementioned transition model 340 from training data. More specifically, training begins by using a feature extraction module 300 to extract features from the contents of the source and destination documents 140 used for training.
9. Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANGELICA RUIZ whose telephone number is (571)270-3158. The examiner can normally be reached M-F 10:00 am to 6:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Boris Gorney can be reached at (571) 270-5626. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANGELICA RUIZ/Primary Examiner, Art Unit 2154 December 27, 2025