DETAILED ACTION
This communication is in response to the Amendments and Arguments filed on 07/21/2025.
Claim 8 has been canceled by the Applicant.
Claim 21 has been newly added by the Applicant.
Claim(s) 1-7 and 9-21 are pending and have been examined. Hence, this action has been made FINAL.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments and Amendments
Amendments to the claims by the Applicant have been considered and addressed below.
With respect to the Drawing Objections and 35 USC § 101, 102, and 103 rejections, the Applicant provides several arguments in which the Examiner will respond accordingly, below.
Drawing Objections
Arguments in page 31 of the Remarks filed on 07/21/2025
Examiner’s Response to Arguments:
Applicant’s arguments with respect to the drawing objections have been fully considered and are persuasive. The drawing objections have been withdrawn.
35 USC § 101 rejection(s)
Arguments in page 31 of the Remarks filed on 07/21/2025
Examiner’s Response to Arguments:
Arguments and amendments have been considered but these are not persuasive.
For more details, please refer to updated 35 U.S.C. § 101 rejections for claims 1-7 and 9-21 below.
35 USC § 103 rejection(s)
Arguments in pages 31 of the Remarks filed on 07/21/2025
Examiner’s Response to Arguments:
Applicant’s arguments with respect to claim(s) 1 and 19-20 under 35 U.S.C. § 102 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of Zhang et al. (US 20190366153 A1).
For more details, please refer to updated 35 U.S.C. § 103 rejections for claims 1-7 and 9-21 below.
Double Patenting
Arguments in pages 32 of the Remarks filed on 07/21/2025
Examiner’s Response to Arguments:
Request for the Double Patenting rejections to be held in abeyance granted.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-7 and 9-21 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of co-pending patent application (U.S. Patent Application No. 17/976,804) in view of Zhang et al. (US 20190366153 A1).
The claims of the issued patent are similar in scope than that of the instant application. However, the claims of the co-pending patent application (U.S. Patent Application No. US 17/976,804) do not explicitly teach wherein the method, as presented in the instant application independent claims, comprise:
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data;
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object;
receiving an indication of a first group of two or more events in the physical world caused by the first physical object;
receiving an indication of a second group of two or more events in the physical world caused by the second physical object;
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object;
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object;
Zhang et al. does teach wherein the method further comprises:
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data (see ¶ [0085]: “In process step 210, objects of interests are detected from frames of the input video. In particular, one or more convolutional neural networks (CNN) may be applied to identify desired objects including balls and players in the input video, and the detected objects are passed as input 215 to process step 220. Each CNN module may be trained using one or more prior input videos. In individual training sessions, only a single player is present, although multiple balls may be moving through the court if a basketball shooting machine is used. In multiple-player training sessions or games, multiple players and multiple balls may be present. A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”);
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object (see Fig. 9B (players: 904-905, ball: 902, and basket/hoop: 980), ¶ [0085] citation as in limitation above. More specifically: “…A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”, and ¶ [0154]: “Embodiments of the present invention may first detect the ball, determine a corresponding trajectory, then trace the ball trajectory to see if it ends in a shot attempt. For example, a box 902 in FIG. 9A represents a ball extraction result with confidence value of 1.000. Trajectory 903, represented as a dotted curve in FIG. 9A, may be reconstructed directly from a ball flow comprising a sequence of ball objects, or be generated by interpolating and/or extrapolating several known ball positions in air. Trajectory 903 represents a pass from player 905 to player 904, where the ball does not move above any of these two player's upper bodies, or come close to basket 980. By comparison, trajectory 963 in FIG. 9G corresponds to a shot attempt by shooter 904. Once a ball flow or trajectory such as 963 is determined, the ball flow can be examined to determine whether the ball has been thrown from the shooter's upper body upward, and if so, declare it as a shot attempt.”
Here, the first physical object is analogous to player 904, while the second physical object is analogous to player 905.);
receiving an indication of a first group of two or more events in the physical world caused by the first physical object (see ¶ [0020 and 0099-0100]: “[0020] In some embodiments, the shot event is selected from the group consisting of dribble event, jump event, catch-ball event, ball-leave-hand event, one-two leg jump, shooter's foot-on-ground movement, and the shot type is selected from the group consisting of layup, regular shot, dribble-pull-up, off-the-move, and catch-and-shoot. [0073] FIG. 1B is a flow diagram 190 providing a process overview of using a mobile device-based NEX system 150 to generate shot analytics and statistics, according to one embodiment of the present invention. This exemplary process takes as inputs a video segment or video stream, and/or a shooter's location in any given frame of the video input. Through new and novel methods for computer vision and algorithmic analysis, systems and devices implemented according to embodiments of the present invention extract various shot analytics, including, but are not limited to, shot type, release time, release angle, shooter body bend angle, leg bend ratio, moving speed and direction, and height of a jump event. The input video may be a live-stream, or an off-line recording, and may be a single perspective video, also known as a monocular video. [0099] With filtered flow and shot information 315, the NEX system may apply the remaining process steps in FIG. 3 to determine one or more shot events occurring before the ball-from-shooter time, and to generate one or more shot analytics 185 based on the one or more shot events, the shooter posture flow, and the related ball flow. In this disclosure, a “shot event” refers to player actions leading up to a shot attempt. That is, a shot event describes player movements before the ball leaves the shooter's hand in a shot attempt. A shot event may occur right before a shot is launched, or some time shortly before the shot is launched. [0100] In process step 320 shown in FIG. 3, several exemplary shot events are detected, for example, a dribble event, a jump event, a catch-ball event, as well as shooter movement in image space. Detected shot events, shooter movement in image space, shooter posture flow, and ball-shoot-from-hand time are used as input 325 to further processing steps 329, 330 and optionally 331, to determine one or more shot analytics.”
Here, the Examiner notes that the indication of a first group of two or more events is read by disclosures in Zhang et al. regarding events associated with player 904 and the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.).);
receiving an indication of a second group of two or more events in the physical world caused by the second physical object (see ¶ [0020 and 0099-0100] citations as in limitation above and further ¶ [0150]: “In table 810, raw information is divided into ball information 812, shooter information 814, events information 816, and scene information 818. For balls extracted from the input video, one or more ball flow and trajectories may be identified, and shot attempts may be determined based on the ball trajectories and their positions relative to the hoop. For the shooter, pose information may be determined from, for example, 18 key points on the body. Following a shot attempt trajectory, shooter poses may be detected in the region around the ball, and tracked as shooter poses. In some embodiments, more than one player may be present, and shooter information 814 may refer to player pose information and player posture flow as discussed with reference to FIGS. 1B to 4. In addition, shooter information 814 may be correlated with ball information 812 to determine different shot events such as ball-leave-hand, jump, dribble, and catch-ball events. Scene information 818 includes how hoop, court, and other relevant objects of interests are placed within the image domain, including hoop detection information and how court is placed in the image. Such scene information may be combined with other ball, shooter, and events information to generate shot analytics and/or game analytics, such as determining whether a shot is a 3-pointer or not.”
Here, the Examiner notes that the indication of a second group of two or more events is read by disclosures in Zhang et al. regarding events associated with the second player 905 and the ball OR by disclosures of other scene information (e.g., hoop, court or other relevant objects present in the image).);
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the determining to include in textual content a description based on the first group of two or more events is read by disclosures in Zhang et al. regarding shot analytics or the events associated with player 904 managing the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the Examiner notes that no information regarding the additional player 905 OR scene information (e.g., hoop) are included in the shot analytics as disclosed in Zhang et al., wherein only shot analytics or the events associated with player 904 managing the ball are included (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
U.S. Patent Application No. US 17/976,804 in view of Zhang et al. (US 20190366153 A1) are considered to be analogous to the claimed invention because they are in the same field of endeavor in text/information generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified U.S. Patent Application No. US 17/976,804 to incorporate the teachings of Zhang et al. of calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data; identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object; receiving an indication of a first group of two or more events in the physical world caused by the first physical object; receiving an indication of a second group of two or more events in the physical world caused by the second physical object; based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object; based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object; which provides the benefit optimizing the feature extraction process to not only reduce the overall computational complexity but also improve the achievable accuracy by tailoring to the specific small input and ball detection goal.([0134] of Zhang et al.).
Claims 1-7 and 9-21 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of co-pending patent application (U.S. Patent Application No. 17/976,806) in view of Zhang et al. (US 20190366153 A1).
The claims of the issued patent are similar in scope than that of the instant application. However, the claims of the co-pending patent application (U.S. Patent Application No. US 17/976,806) do not explicitly teach wherein the method, as presented in the instant application independent claims, comprise:
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data;
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object;
receiving an indication of a first group of two or more events in the physical world caused by the first physical object;
receiving an indication of a second group of two or more events in the physical world caused by the second physical object;
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object;
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object;
Zhang et al. does teach wherein the method further comprises:
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data (see ¶ [0085]: “In process step 210, objects of interests are detected from frames of the input video. In particular, one or more convolutional neural networks (CNN) may be applied to identify desired objects including balls and players in the input video, and the detected objects are passed as input 215 to process step 220. Each CNN module may be trained using one or more prior input videos. In individual training sessions, only a single player is present, although multiple balls may be moving through the court if a basketball shooting machine is used. In multiple-player training sessions or games, multiple players and multiple balls may be present. A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”);
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object (see Fig. 9B (players: 904-905, ball: 902, and basket/hoop: 980), ¶ [0085] citation as in limitation above. More specifically: “…A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”, and ¶ [0154]: “Embodiments of the present invention may first detect the ball, determine a corresponding trajectory, then trace the ball trajectory to see if it ends in a shot attempt. For example, a box 902 in FIG. 9A represents a ball extraction result with confidence value of 1.000. Trajectory 903, represented as a dotted curve in FIG. 9A, may be reconstructed directly from a ball flow comprising a sequence of ball objects, or be generated by interpolating and/or extrapolating several known ball positions in air. Trajectory 903 represents a pass from player 905 to player 904, where the ball does not move above any of these two player's upper bodies, or come close to basket 980. By comparison, trajectory 963 in FIG. 9G corresponds to a shot attempt by shooter 904. Once a ball flow or trajectory such as 963 is determined, the ball flow can be examined to determine whether the ball has been thrown from the shooter's upper body upward, and if so, declare it as a shot attempt.”
Here, the first physical object is analogous to player 904, while the second physical object is analogous to player 905.);
receiving an indication of a first group of two or more events in the physical world caused by the first physical object (see ¶ [0020 and 0099-0100]: “[0020] In some embodiments, the shot event is selected from the group consisting of dribble event, jump event, catch-ball event, ball-leave-hand event, one-two leg jump, shooter's foot-on-ground movement, and the shot type is selected from the group consisting of layup, regular shot, dribble-pull-up, off-the-move, and catch-and-shoot. [0073] FIG. 1B is a flow diagram 190 providing a process overview of using a mobile device-based NEX system 150 to generate shot analytics and statistics, according to one embodiment of the present invention. This exemplary process takes as inputs a video segment or video stream, and/or a shooter's location in any given frame of the video input. Through new and novel methods for computer vision and algorithmic analysis, systems and devices implemented according to embodiments of the present invention extract various shot analytics, including, but are not limited to, shot type, release time, release angle, shooter body bend angle, leg bend ratio, moving speed and direction, and height of a jump event. The input video may be a live-stream, or an off-line recording, and may be a single perspective video, also known as a monocular video. [0099] With filtered flow and shot information 315, the NEX system may apply the remaining process steps in FIG. 3 to determine one or more shot events occurring before the ball-from-shooter time, and to generate one or more shot analytics 185 based on the one or more shot events, the shooter posture flow, and the related ball flow. In this disclosure, a “shot event” refers to player actions leading up to a shot attempt. That is, a shot event describes player movements before the ball leaves the shooter's hand in a shot attempt. A shot event may occur right before a shot is launched, or some time shortly before the shot is launched. [0100] In process step 320 shown in FIG. 3, several exemplary shot events are detected, for example, a dribble event, a jump event, a catch-ball event, as well as shooter movement in image space. Detected shot events, shooter movement in image space, shooter posture flow, and ball-shoot-from-hand time are used as input 325 to further processing steps 329, 330 and optionally 331, to determine one or more shot analytics.”
Here, the Examiner notes that the indication of a first group of two or more events is read by disclosures in Zhang et al. regarding events associated with player 904 and the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.).);
receiving an indication of a second group of two or more events in the physical world caused by the second physical object (see ¶ [0020 and 0099-0100] citations as in limitation above and further ¶ [0150]: “In table 810, raw information is divided into ball information 812, shooter information 814, events information 816, and scene information 818. For balls extracted from the input video, one or more ball flow and trajectories may be identified, and shot attempts may be determined based on the ball trajectories and their positions relative to the hoop. For the shooter, pose information may be determined from, for example, 18 key points on the body. Following a shot attempt trajectory, shooter poses may be detected in the region around the ball, and tracked as shooter poses. In some embodiments, more than one player may be present, and shooter information 814 may refer to player pose information and player posture flow as discussed with reference to FIGS. 1B to 4. In addition, shooter information 814 may be correlated with ball information 812 to determine different shot events such as ball-leave-hand, jump, dribble, and catch-ball events. Scene information 818 includes how hoop, court, and other relevant objects of interests are placed within the image domain, including hoop detection information and how court is placed in the image. Such scene information may be combined with other ball, shooter, and events information to generate shot analytics and/or game analytics, such as determining whether a shot is a 3-pointer or not.”
Here, the Examiner notes that the indication of a second group of two or more events is read by disclosures in Zhang et al. regarding events associated with the second player 905 and the ball OR by disclosures of other scene information (e.g., hoop, court or other relevant objects present in the image).);
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the determining to include in textual content a description based on the first group of two or more events is read by disclosures in Zhang et al. regarding shot analytics or the events associated with player 904 managing the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the Examiner notes that no information regarding the additional player 905 OR scene information (e.g., hoop) are included in the shot analytics as disclosed in Zhang et al., wherein only shot analytics or the events associated with player 904 managing the ball are included (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
U.S. Patent Application No. US 17/976,806 in view of Zhang et al. (US 20190366153 A1) are considered to be analogous to the claimed invention because they are in the same field of endeavor in text/information generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified U.S. Patent Application No. US 17/976,806 to incorporate the teachings of Zhang et al. of calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data; identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object; receiving an indication of a first group of two or more events in the physical world caused by the first physical object; receiving an indication of a second group of two or more events in the physical world caused by the second physical object; based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object; based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object; which provides the benefit optimizing the feature extraction process to not only reduce the overall computational complexity but also improve the achievable accuracy by tailoring to the specific small input and ball detection goal.([0134] of Zhang et al.).
Claims 1-7 and 9-21 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of co-pending patent application (U.S. Patent Application No. 17/976,807) in view of Zhang et al. (US 20190366153 A1).
The claims of the issued patent are similar in scope than that of the instant application. However, the claims of the co-pending patent application (U.S. Patent Application No. US 17/976,807) do not explicitly teach wherein the method, as presented in the instant application independent claims, comprise:
receiving an indication of a plurality of objects, the plurality of objects includes at least a first physical object and a second physical object;
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data;
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object;
receiving an indication of a first group of two or more events in the physical world caused by the first physical object;
receiving an indication of a second group of two or more events in the physical world caused by the second physical object;
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object;
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object;
Zhang et al. does teach wherein the method further comprises:
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data (see ¶ [0085]: “In process step 210, objects of interests are detected from frames of the input video. In particular, one or more convolutional neural networks (CNN) may be applied to identify desired objects including balls and players in the input video, and the detected objects are passed as input 215 to process step 220. Each CNN module may be trained using one or more prior input videos. In individual training sessions, only a single player is present, although multiple balls may be moving through the court if a basketball shooting machine is used. In multiple-player training sessions or games, multiple players and multiple balls may be present. A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”);
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object (see Fig. 9B (players: 904-905, ball: 902, and basket/hoop: 980), ¶ [0085] citation as in limitation above. More specifically: “…A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”, and ¶ [0154]: “Embodiments of the present invention may first detect the ball, determine a corresponding trajectory, then trace the ball trajectory to see if it ends in a shot attempt. For example, a box 902 in FIG. 9A represents a ball extraction result with confidence value of 1.000. Trajectory 903, represented as a dotted curve in FIG. 9A, may be reconstructed directly from a ball flow comprising a sequence of ball objects, or be generated by interpolating and/or extrapolating several known ball positions in air. Trajectory 903 represents a pass from player 905 to player 904, where the ball does not move above any of these two player's upper bodies, or come close to basket 980. By comparison, trajectory 963 in FIG. 9G corresponds to a shot attempt by shooter 904. Once a ball flow or trajectory such as 963 is determined, the ball flow can be examined to determine whether the ball has been thrown from the shooter's upper body upward, and if so, declare it as a shot attempt.”
Here, the first physical object is analogous to player 904, while the second physical object is analogous to player 905.);
receiving an indication of a first group of two or more events in the physical world caused by the first physical object (see ¶ [0020 and 0099-0100]: “[0020] In some embodiments, the shot event is selected from the group consisting of dribble event, jump event, catch-ball event, ball-leave-hand event, one-two leg jump, shooter's foot-on-ground movement, and the shot type is selected from the group consisting of layup, regular shot, dribble-pull-up, off-the-move, and catch-and-shoot. [0073] FIG. 1B is a flow diagram 190 providing a process overview of using a mobile device-based NEX system 150 to generate shot analytics and statistics, according to one embodiment of the present invention. This exemplary process takes as inputs a video segment or video stream, and/or a shooter's location in any given frame of the video input. Through new and novel methods for computer vision and algorithmic analysis, systems and devices implemented according to embodiments of the present invention extract various shot analytics, including, but are not limited to, shot type, release time, release angle, shooter body bend angle, leg bend ratio, moving speed and direction, and height of a jump event. The input video may be a live-stream, or an off-line recording, and may be a single perspective video, also known as a monocular video. [0099] With filtered flow and shot information 315, the NEX system may apply the remaining process steps in FIG. 3 to determine one or more shot events occurring before the ball-from-shooter time, and to generate one or more shot analytics 185 based on the one or more shot events, the shooter posture flow, and the related ball flow. In this disclosure, a “shot event” refers to player actions leading up to a shot attempt. That is, a shot event describes player movements before the ball leaves the shooter's hand in a shot attempt. A shot event may occur right before a shot is launched, or some time shortly before the shot is launched. [0100] In process step 320 shown in FIG. 3, several exemplary shot events are detected, for example, a dribble event, a jump event, a catch-ball event, as well as shooter movement in image space. Detected shot events, shooter movement in image space, shooter posture flow, and ball-shoot-from-hand time are used as input 325 to further processing steps 329, 330 and optionally 331, to determine one or more shot analytics.”
Here, the Examiner notes that the indication of a first group of two or more events is read by disclosures in Zhang et al. regarding events associated with player 904 and the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.).);
receiving an indication of a second group of two or more events in the physical world caused by the second physical object (see ¶ [0020 and 0099-0100] citations as in limitation above and further ¶ [0150]: “In table 810, raw information is divided into ball information 812, shooter information 814, events information 816, and scene information 818. For balls extracted from the input video, one or more ball flow and trajectories may be identified, and shot attempts may be determined based on the ball trajectories and their positions relative to the hoop. For the shooter, pose information may be determined from, for example, 18 key points on the body. Following a shot attempt trajectory, shooter poses may be detected in the region around the ball, and tracked as shooter poses. In some embodiments, more than one player may be present, and shooter information 814 may refer to player pose information and player posture flow as discussed with reference to FIGS. 1B to 4. In addition, shooter information 814 may be correlated with ball information 812 to determine different shot events such as ball-leave-hand, jump, dribble, and catch-ball events. Scene information 818 includes how hoop, court, and other relevant objects of interests are placed within the image domain, including hoop detection information and how court is placed in the image. Such scene information may be combined with other ball, shooter, and events information to generate shot analytics and/or game analytics, such as determining whether a shot is a 3-pointer or not.”
Here, the Examiner notes that the indication of a second group of two or more events is read by disclosures in Zhang et al. regarding events associated with the second player 905 and the ball OR by disclosures of other scene information (e.g., hoop, court or other relevant objects present in the image).);
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the determining to include in textual content a description based on the first group of two or more events is read by disclosures in Zhang et al. regarding shot analytics or the events associated with player 904 managing the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the Examiner notes that no information regarding the additional player 905 OR scene information (e.g., hoop) are included in the shot analytics as disclosed in Zhang et al., wherein only shot analytics or the events associated with player 904 managing the ball are included (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
U.S. Patent Application No. US 17/976,807 in view of Zhang et al. (US 20190366153 A1) are considered to be analogous to the claimed invention because they are in the same field of endeavor in text/information generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified U.S. Patent Application No. US 17/976,807 to incorporate the teachings of Zhang et al. of calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data; identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object; receiving an indication of a first group of two or more events in the physical world caused by the first physical object; receiving an indication of a second group of two or more events in the physical world caused by the second physical object; based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object; based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object; which provides the benefit optimizing the feature extraction process to not only reduce the overall computational complexity but also improve the achievable accuracy by tailoring to the specific small input and ball detection goal.([0134] of Zhang et al.).
Claims 1-7 and 9-21 are provisionally rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of co-pending patent application (U.S. Patent Application No. 17/976,808) in view of Zhang et al. (US 20190366153 A1).
The claims of the issued patent are similar in scope than that of the instant application. However, the claims of the co-pending patent application (U.S. Patent Application No. US 17/976,808) do not explicitly teach wherein the method, as presented in the instant application independent claims, comprise:
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data;
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object;
receiving an indication of a first group of two or more events in the physical world caused by the first physical object;
receiving an indication of a second group of two or more events in the physical world caused by the second physical object;
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object;
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object;
Zhang et al. does teach wherein the method further comprises:
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data (see ¶ [0085]: “In process step 210, objects of interests are detected from frames of the input video. In particular, one or more convolutional neural networks (CNN) may be applied to identify desired objects including balls and players in the input video, and the detected objects are passed as input 215 to process step 220. Each CNN module may be trained using one or more prior input videos. In individual training sessions, only a single player is present, although multiple balls may be moving through the court if a basketball shooting machine is used. In multiple-player training sessions or games, multiple players and multiple balls may be present. A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”);
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object (see Fig. 9B (players: 904-905, ball: 902, and basket/hoop: 980), ¶ [0085] citation as in limitation above. More specifically: “…A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”, and ¶ [0154]: “Embodiments of the present invention may first detect the ball, determine a corresponding trajectory, then trace the ball trajectory to see if it ends in a shot attempt. For example, a box 902 in FIG. 9A represents a ball extraction result with confidence value of 1.000. Trajectory 903, represented as a dotted curve in FIG. 9A, may be reconstructed directly from a ball flow comprising a sequence of ball objects, or be generated by interpolating and/or extrapolating several known ball positions in air. Trajectory 903 represents a pass from player 905 to player 904, where the ball does not move above any of these two player's upper bodies, or come close to basket 980. By comparison, trajectory 963 in FIG. 9G corresponds to a shot attempt by shooter 904. Once a ball flow or trajectory such as 963 is determined, the ball flow can be examined to determine whether the ball has been thrown from the shooter's upper body upward, and if so, declare it as a shot attempt.”
Here, the first physical object is analogous to player 904, while the second physical object is analogous to player 905.);
receiving an indication of a first group of two or more events in the physical world caused by the first physical object (see ¶ [0020 and 0099-0100]: “[0020] In some embodiments, the shot event is selected from the group consisting of dribble event, jump event, catch-ball event, ball-leave-hand event, one-two leg jump, shooter's foot-on-ground movement, and the shot type is selected from the group consisting of layup, regular shot, dribble-pull-up, off-the-move, and catch-and-shoot. [0073] FIG. 1B is a flow diagram 190 providing a process overview of using a mobile device-based NEX system 150 to generate shot analytics and statistics, according to one embodiment of the present invention. This exemplary process takes as inputs a video segment or video stream, and/or a shooter's location in any given frame of the video input. Through new and novel methods for computer vision and algorithmic analysis, systems and devices implemented according to embodiments of the present invention extract various shot analytics, including, but are not limited to, shot type, release time, release angle, shooter body bend angle, leg bend ratio, moving speed and direction, and height of a jump event. The input video may be a live-stream, or an off-line recording, and may be a single perspective video, also known as a monocular video. [0099] With filtered flow and shot information 315, the NEX system may apply the remaining process steps in FIG. 3 to determine one or more shot events occurring before the ball-from-shooter time, and to generate one or more shot analytics 185 based on the one or more shot events, the shooter posture flow, and the related ball flow. In this disclosure, a “shot event” refers to player actions leading up to a shot attempt. That is, a shot event describes player movements before the ball leaves the shooter's hand in a shot attempt. A shot event may occur right before a shot is launched, or some time shortly before the shot is launched. [0100] In process step 320 shown in FIG. 3, several exemplary shot events are detected, for example, a dribble event, a jump event, a catch-ball event, as well as shooter movement in image space. Detected shot events, shooter movement in image space, shooter posture flow, and ball-shoot-from-hand time are used as input 325 to further processing steps 329, 330 and optionally 331, to determine one or more shot analytics.”
Here, the Examiner notes that the indication of a first group of two or more events is read by disclosures in Zhang et al. regarding events associated with player 904 and the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.).);
receiving an indication of a second group of two or more events in the physical world caused by the second physical object (see ¶ [0020 and 0099-0100] citations as in limitation above and further ¶ [0150]: “In table 810, raw information is divided into ball information 812, shooter information 814, events information 816, and scene information 818. For balls extracted from the input video, one or more ball flow and trajectories may be identified, and shot attempts may be determined based on the ball trajectories and their positions relative to the hoop. For the shooter, pose information may be determined from, for example, 18 key points on the body. Following a shot attempt trajectory, shooter poses may be detected in the region around the ball, and tracked as shooter poses. In some embodiments, more than one player may be present, and shooter information 814 may refer to player pose information and player posture flow as discussed with reference to FIGS. 1B to 4. In addition, shooter information 814 may be correlated with ball information 812 to determine different shot events such as ball-leave-hand, jump, dribble, and catch-ball events. Scene information 818 includes how hoop, court, and other relevant objects of interests are placed within the image domain, including hoop detection information and how court is placed in the image. Such scene information may be combined with other ball, shooter, and events information to generate shot analytics and/or game analytics, such as determining whether a shot is a 3-pointer or not.”
Here, the Examiner notes that the indication of a second group of two or more events is read by disclosures in Zhang et al. regarding events associated with the second player 905 and the ball OR by disclosures of other scene information (e.g., hoop, court or other relevant objects present in the image).);
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the determining to include in textual content a description based on the first group of two or more events is read by disclosures in Zhang et al. regarding shot analytics or the events associated with player 904 managing the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the Examiner notes that no information regarding the additional player 905 OR scene information (e.g., hoop) are included in the shot analytics as disclosed in Zhang et al., wherein only shot analytics or the events associated with player 904 managing the ball are included (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
U.S. Patent Application No. US 17/976,808 in view of Zhang et al. (US 20190366153 A1) are considered to be analogous to the claimed invention because they are in the same field of endeavor in text/information generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified U.S. Patent Application No. US 17/976,808 to incorporate the teachings of Zhang et al. of calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data; identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object; receiving an indication of a first group of two or more events in the physical world caused by the first physical object; receiving an indication of a second group of two or more events in the physical world caused by the second physical object; based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object; based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object; which provides the benefit optimizing the feature extraction process to not only reduce the overall computational complexity but also improve the achievable accuracy by tailoring to the specific small input and ball detection goal.([0134] of Zhang et al.).
These are provisional nonstatutory double patenting rejection(s) because patentably indistinct claim(s) have not been patented yet.
Please see the claim mapping as well as the claim mappings for the individual claims in the tables below.
Instant Application
17/976,812
Copending
U.S. Application No. 17/976,804
Copending
U.S. Application No. 17/976,806
Copending
U.S. Application No. 17/976,807
Copending
U.S. Application No. 17/976,808
Claim mapping
1, 7, 9, and 19-20
1, 6 and 19-20
1, 6 and 19-20
1, 3-4, 6, 16, 19-20
1, 2-3, 5-6, and 19-20
2
4
4
-
5
3
15
15
-
-
4
-
-
-
-
5
-
-
-
-
6
7, 14, 18
7, 14, 18
13
10
10
-
-
-
11
5
5
12
12
12
5
5
12
12
13
8
8
9
14
14
9
9
10
15
15
17
17
11
16
16
-
-
-
-
17
-
-
-
-
18
-
-
-
-
21
-
-
-
-
Instant Application
17/976,812
Copending
U.S. Application No. 17/976,804
Copending
U.S. Application No. 17/976,806
Copending
U.S. Application No. 17/976,807
Copending
U.S. Application No. 17/976,808
Claim 1:
Claim 1:
Claim 1:
Claim 1:
Claim 1:
1. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generating a textual content selectively reporting objects, the operations comprising:
1. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for analyzing audio data to generate a textual content reporting objects, the operations comprising:
1. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for analyzing visual data to generate a textual content reporting events, the operations comprising:
1. (Currently Amended) A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generating a textual content selectively reporting events, the operations comprising:
1. A non-transitory computer readable medium storing a software program comprising data and computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generating a textual content reporting events, the operations comprising:
receiving an indication of a plurality of physical objects, the plurality of physical objects includes at least a first physical object and a second physical object;
receiving an indication of a group of one or more objects associated with the event;
receiving an indication of a first group of one or more events associated with the first physical object;
receiving an indication of a plurality of events;
receiving an indication of an event;
receiving an indication of a second group of one or more events associated with the second physical object, the second group of one or more events includes at least one event not included in the first group of one or more events;
based on the first group of one or more events, determining to include in a textual content a description based on the first group of one or more events of the first physical object;
based on the second group of one or more events, determining not to include in the textual content any description based on the second group of one or more events of the second physical object;
Claim 1:
Claim 1:
Claim 1:
Claim 1:
Claim 1:
for each event of the first group of one or more events,
receiving data associated with the event;
for each object of the plurality of objects,
for each event of the plurality of events,
for each event of the plurality of events,
for each object of the group of one or more objects,
receiving data associated with the object;
analyzing the data associated with the first group of one or more events to
generate a particular description of the first physical object, the particular description of the first physical object is based on the first group of one or more events;
analyzing data associated with the object to select an adjective, and
generating a description of the object that includes the adjective;
analyzing data associated with the event to select an adjective, and
generating a description of the event that includes the adjective;
analyzing data associated with the particular event to
generate a description of the particular event;
analyzing the data associated with the group of one or more objects to select an adjective;
generating a particular description of the event, the particular description is based on the group of one or more objects, the particular description includes the selected adjective;
generating the textual content, the textual content includes the particular description of the first physical object and does not include any description based on the second group of one or more events of the second physical object; and
generating a textual content that includes the generated descriptions of the plurality of objects; and
generating a textual content that includes the generated descriptions of the plurality of events; and
using a generative model to generate a textual content that includes the description of the group of two or more events and the description of the particular event, wherein for at least one specific event of the group of two or more events, the textual content does not include information identifying the specific event; and
generating a textual content, the textual content includes the particular description; and
providing the generated textual content.
providing the generated textual content.
providing the generated textual content.
providing the generated textual content.
providing the generated textual content.
determining a quantity associated with the group of two or more events;
generating a description of the group of two or more events, the description of the group of two or more events includes an indication of the quantity associated with the group of two or more events;
Claim 1:
Claim 1:
Claim 3:
receiving image data;
analyzing the data associated with the first group of two or more events to generate a particular description of the first physical object;
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data;
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object;
receiving image data captured using at least one image sensor;
analyzing the image data to identify a plurality of events;
3. (Currently Amended) The non-transitory computer readable medium of claim 1,
wherein each event in the group of two or more events is associated with image data, and the operations further comprise:
for each event in the group of two or more events,
analyzing the image data associated with the event to determine [[data]]information associated with the event; and
determining the quantity associated with the group of two or more events based on the [[data]]information associated with the events in the group of two or more events.
Claim 6:
Claim 1:
Claim 4:
receiving audio data captured from the environment using at least one audio sensor;
accessing synchronization data configured to enable synchronization of the image data and the audio data;
using the
synchronization data to identify a first at least one portion of the audio data associated with the first physical object and to identify a second at least one portion of audio data associated with the second physical object;
analyzing the first at least one portion of the audio data to detect events of the first group of two or more events; and
analyzing the second at least one portion of the audio data to detect events of the second group of two or more events
receiving audio data captured using at least one audio sensor;
analyzing the audio data to identify a plurality of objects;
4. (Currently Amended) The non-transitory computer readable medium of claim 1,
wherein each event in the group of two or more events is associated with audio data, and the operations further comprise:
for each event in the group of two or more events,
analyzing the audio data associated with the event to determine [[data]]information associated with the event; and
determining the quantity associated with the group of two or more events based on the [[data]] information associated with the events in the group of two or more events.
Claim 9:
Claim 1:
9. The non-transitory computer readable medium of claim 1,
wherein the operations further comprise:
determining a first magnitude associated with the first group of two or more events;
based on the first magnitude, determining to include in the textual content the description based on the first group of two or more events of the first physical object;
determining a second magnitude associated with the second group of two or more events; and
based on the second magnitude, determining not to include in the textual content any description based on the second group of two or more events of the second physical object.
determining a respective magnitude associated with the event:
based on the determining a respective magnitude associated with the event:
magnitudes associated with the events in the plurality of events, identifying a group of two or more events of the plurality of events, the group of two or more events does not include at least a particular event of the plurality of events;
*Note: Main differences between instant application and issued patent/application are underlined/strikethrough.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7 and 9-21 rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. More specifically directed to the abstract idea grouping of: mental process and/or mathematical concept.
The independent claim(s) 1, 19-20 recite(s):
receiving image data;
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data:
receiving an indication of a plurality of objects, the plurality of physical objects includes at least a first physical object and a second physical object;
receiving an indication of a first group of one or more events associated with the first physical object;
receiving an indication of a second group of one or more events associated with the second physical object, the second group of one or more events includes at least one event not included in the first group of one or more events;
based on the first group of one or more events, determining to include in a textual content a description based on the first group of one or more events of the first physical object;
based on the second group of one or more events, determining not to include in the textual content any description based on the second group of one or more events of the second physical object;
for each event of the first group of one or more events, receiving data associated with the event;
analyzing the data associated with the first group of one or more events to generate a particular description of the first physical object, the particular description of the first physical object is based on the first group of one or more events;
generating the textual content, the textual content includes the particular description of the first physical object and does not include any description based on the second group of one or more events of the second physical object; and
providing the generated textual content.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving a printed image;
Calculating (i.e., mathematical concept) using convolution (i.e., set of predefined steps) for analyzing the image;
Receiving indication of a plurality of objects;
Receiving indication of a plurality of events associated with an object;
Receiving indication of a second group of plurality of events associated with another object;
Based on the group of events determining what information to include in a text (e.g., associated to first object);
Based on the group of events determining what information not to include in a text (e.g., associated to second object);
Receiving data associated to the events;
Analyzing data to generate description of firs object based on the events;
Writing down the textual content including description of the first object;
Providing or sharing said text.
This judicial exception is not integrated into a practical application because for example: claim 1 recites “non-transitory computer readable medium”, “software program”, “processor” and claim 19 recites “processing unit”. As an example, in [045] of the as filed specification, it is disclosed that “…The terms "computer", "processor", "controller", "processing unit", "computing unit", and " processing module" should be expansively construed to cover any kind of electronic device, component or unit with data processing capabilities, including, by way of non- limiting example, a personal computer, a wearable computer, a tablet, a smartphone, a server, a computing system, a cloud computing platform, a communication device, a processor (for example, digital signal processor (DSP), an image signal processor (ISR), a microcontroller, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a central processing unit (CPA), a graphics processing unit (GPU), a visual processing unit (VPU), and so on), possibly with embedded memory, a single core processor, a multi core processor, a core within a processor, any other electronic computing device, or any combination of the above.”. Therefore, a general-purpose computer or computing device is described and mainly used as an application thereof. Accordingly, these additional elements do not integrate the abstract idea into a practical idea because it does not impose any meaningful limits on practicing the abstract idea.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of using a computer is listed as a general computing device as noted. The claim is not patent eligible.
With respect to claim 2, the claim(s) recite:
2. The non-transitory computer readable medium of claim 1, wherein the determination to include in the textual content the description based on the first group of two or more events of the first physical object is further based on the second group of two or more events, and the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is further based on the first group of two or more events.
This reads on a human (e.g., mentally and/or using pen and paper):
Based on the group of events determining what information to include or not in a text (e.g., associated to second object);
No additional limitations are present.
With respect to claim 3, the claim(s) recite:
3. The non-transitory computer readable medium of claim 1, wherein the plurality of physical objects further includes a third object, the third object is associated with a third group of two or more events, the third group of two or more events includes at least one event not included in the first group and the second group, the determination to include in the textual content the description based on the first group of two or more events of the first physical object is further based on the third group of two or more events, and the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is further based on the third group of two or more events.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the plurality of objects includes a third object associated with a third plurality of events associated with another object;
Based on the group of events determining what information to include in a text (e.g., associated to first object) and the third object;
Based on the group of events determining what information not to include in a text (e.g., associated to second object) and the third object;
No additional limitations are present.
With respect to claim 4, the claim(s) recite:
4. The non-transitory computer readable medium of claim 1, wherein the determination to include in the textual content the description based on the first group of two or more events of the first physical object is further based on a type of the first physical object and a type of the second physical object, and the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is further based on the type of the first physical object and the type of the second physical object.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the determination about what information to include in a text (e.g., associated to first object) and the type of first and second objects;
And wherein the determination about what information not to include in a text (e.g., associated to second object) and the type of first and second objects;
No additional limitations are present.
With respect to claim 5, the claim(s) recite:
5. The non-transitory computer readable medium of claim 1, wherein the determination to include in the textual content the description based on the first group of two or more events of the first physical object is further based on a type associated with a specific event of the first group of two or more events, and the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is further based on a type associated with a specific event of the second group of two or more events.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the determination about what information to include in a text (e.g., associated to first object) and the type of a specific event of the plurality of events;
And wherein the determination about what information not to include in a text (e.g., associated to second object) and the type of a specific event of the plurality of events.
No additional limitations are present.
With respect to claim 6, the claim(s) recite:
6. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
receiving audio data captured from the environment using at least one audio sensor;
accessing synchronization data configured to enable synchronization of the first modality data and the second modality data;
using the synchronization data to identify a first at least one portion of the second modality data associated with the first physical object and to identify a second at least one portion of the second modality data associated with the second physical object;
analyzing the first at least one portion of the second modality data to detect events of the first group of two or more events; and
analyzing the second at least one portion of the second modality data to detect events of the second group of two or more events.
This reads on a human (e.g., mentally and/or using pen and paper):
Hearing a voice/utterance from another human(s);
Analyzing/listening the utterance to identify objects;
Analyzing/listening the utterance to identify events;
Analyzing/listening the utterance to detect additional events
Analyzing data to detect objects;
Synchronize or match first and second data using predetermined set of rules;
Access predetermined set of rules to perform the synchronization;
Associate portions of the synchronized data with first and second objects;
Analyze portions of data to detect events of a first group of events;
Analyze portions of data to detect events of a second group of events
This judicial exception is not integrated into a practical application because for example: claim 6 recites “first type of sensor” and “second type of sensor”. As an example, in [045] of the as filed specification (as discussed above), [049] discloses “… Examples of image sensor technologies may include: CCD, CMOS, NMOS, and so forth. 3D sensors may be implemented using different technologies, including: stereo camera, active stereo camera, time of flight camera, structured light camera, radar, range image camera, and so forth.” and [054] discloses “… Some non- limiting examples of such sensors may include image sensors (such as image sensor 260), audio sensors (such as audio sensors 250), motion sensors (such as motion sensor 270), positioning sensors (such as positioning sensors 275), touch sensors, proximity sensors, chemical sensors, temperature sensors, barometers, and so forth. In some examples, log data 108 may include any information recording activities. Some non-limiting examples of such log data may include a digital log file, a hardcopy log file, a handwritten log, an audio log recorded using at least one audio sensor, a visual log or a video log recorded using at least one image sensor, and so forth.”. Therefore, a general-purpose computer or computing device is described and mainly used as an application thereof. Accordingly, these additional elements do not integrate the abstract idea into a practical idea because it does not impose any meaningful limits on practicing the abstract idea.
The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements of using a computer is listed as a general computing device as noted. The claim is not patent eligible.
With respect to claim 7, the claim(s) recite:
7. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
analyzing the image data to detect events of the first group of two or more events; and
analyzing the image data to detect events of the second group of two or more events.
This reads on a human (e.g., mentally and/or using pen and paper):
Analyzing image to identify events;
Analyzing image to detect additional events
No additional limitations are present.
With respect to claim 8, the claim(s) recite:
8. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
receiving audio data;
analyzing the audio data to detect the plurality of physical objects;
analyzing the audio data to detect events of the first group of two or more events; and
analyzing the audio data to detect events of the second group of two or more events.
This reads on a human (e.g., mentally and/or using pen and paper):
Hearing a voice/utterance from another human(s);
Analyzing/listening the utterance to identify objects;
Analyzing/listening the utterance to identify events;
Analyzing/listening the utterance to detect additional events
No additional limitations are present.
With respect to claim 9, the claim(s) recite:
9. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
determining a first magnitude associated with the first group of two or more events;
based on the first magnitude, determining to include in the textual content the description based on the first group of two or more events of the first obj ect;
determining a second magnitude associated with the second group of two or more events; and
based on the second magnitude, determining not to include in the textual content any description based on the second group of two or more events of the second physical object.
This reads on a human (e.g., mentally and/or using pen and paper):
Determining/assigning a value (i.e., magnitude) to the first group of events;
Based on said value determining to include information on text;
Based on a second value determining not to include information on text
No additional limitations are present.
With respect to claim 10, the claim(s) recite:
10. The non-transitory computer readable medium of claim 9, wherein the determination to include in the textual content the description based on the first group of two or more events of the first physical object is further based on the second magnitude, and wherein the determining not to include in the textual content any description based on the second group of two or more events of the second physical object is further based on first magnitude.
This reads on a human (e.g., mentally and/or using pen and paper):
Based on a second value determining to include information on text;
Based on a first value determining not to include information on text
No additional limitations are present.
With respect to claim 11, the claim(s) recite:
11. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
for each event of the first group of two or more events and for each event of the second group of two or more events, determining a mathematical object corresponding to the event in a mathematical space;
basing the determination to include in the textual content the description based on the first group of two or more events of the first physical object on the mathematical objects corresponding to the first group of two or more events; and
basing the determination not to include in the textual content any description based on the second group of two or more events of the second physical object on the mathematical objects corresponding to the second group of two or more events.
This reads on a human (e.g., mentally and/or using pen and paper):
For each event determine a mathematical object (e.g., mathematical equation or symbol);
Based on a mathematical object associated with the first object determining to include information on text;
Based on a mathematical object associated with the second physical object determining not to include information on text
No additional limitations are present.
With respect to claim 12, the claim(s) recite:
12. The non-transitory computer readable medium of claim 11, wherein the operations further comprise:
analyzing the mathematical objects corresponding to the first group of two or more events to determine a first mathematical object in the mathematical space, the first mathematical object differs from any mathematical object of the mathematical objects corresponding to the first group of two or more events;
analyzing the mathematical objects corresponding to the second group of two or more events to determine a second mathematical object in the mathematical space, the second mathematical object differs from any mathematical object of the mathematical objects corresponding to the second group of two or more events;
basing the determination to include in the textual content the description based on the first group of two or more events of the first physical object on the first mathematical object; and
basing the determination not to include in the textual content any description based on the second group of two or more events of the second physical object on the second mathematical object.
This reads on a human (e.g., mentally and/or using pen and paper):
Analyzing the mathematical object(s) (e.g., mathematical equation(s) or symbol(s)) associated with the group of events;
Analyzing the mathematical object(s) (e.g., mathematical equation(s) or symbol(s)) associated with second group of events;
Based on a mathematical object associated with the second group of events determining to include information on text;
Based on a mathematical object associated with the first group of events determining not to include information on text
No additional limitations are present.
With respect to claim 13, the claim(s) recite:
13. The non-transitory computer readable medium of claim 1, wherein the textual content is associated with a writer persona, the determination to include in the textual content the description based on the first group of two or more events of the first physical object is based on the writer persona, and the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is based on the writer persona.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the text content is associated with a persona and the determination to include or not the description based on the events of the first/second objects and the persona.
No additional limitations are present.
With respect to claim 14, the claim(s) recite:
14. The non-transitory computer readable medium of claim 1, wherein the textual content is associated with a prospective audience, the determination to include in the textual content the description based on the first group of two or more events of the first physical object is based on the prospective audience, and the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is based on the prospective audience.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the text content is associated with a prospective audience and the determination to include or not the description based on the events of the first/second objects and the prospective audience.
No additional limitations are present.
With respect to claim 15, the claim(s) recite:
15. The non-transitory computer readable medium of claim 1, wherein the textual content is associated with a topic, the determination to include in the textual content the description based on the first group of two or more events of the first physical object is based on a degree of relevance of the first physical object to the topic, and the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is based on a degree of relevance of the second physical object to the topic.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the text content is associated with a topic and the determination to include or not the description based on the events of the first/second objects and the topic.
No additional limitations are present.
With respect to claim 16, the claim(s) recite:
16. The non-transitory computer readable medium of claim 1, wherein the plurality of physical objects includes at least two additional objects in addition to the first and second physical objects, and the operations further comprise:
selecting a subset of at least one but not all of the at least two additional object;
for each object in the subset, generating a description of the object; and
including the generated descriptions of all objects in the subset in the generated textual content.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the plurality of object includes at least two additional objects;
Selecting one of the additional objects;
Writing down a description of each object;
And including it in text.
No additional limitations are present.
With respect to claim 17, the claim(s) recite:
17. The non-transitory computer readable medium of claim 16, wherein the operations further comprise, for each object of the at least two additional object:
receiving an indication of a group of two or more events associated with the object; and
based on the group of two or more events associated with the object, determining whether to include the object in the subset.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving an indication associated to events associated to the additional objects
Determining if to include or not the object.
No additional limitations are present.
With respect to claim 18, the claim(s) recite:
18. The non-transitory computer readable medium of claim 16, wherein the operations further comprise, for each object of the subset:
receiving an indication of a group of two or more events associated with the object;
for each event of the group of two or more events associated with the object, receiving data associated with the event; and
analyzing the data associated with the group of two or more events associated with the object to generate the description based on the group of two or more events associated with the object of the object, thereby generating the description of the object.
This reads on a human (e.g., mentally and/or using pen and paper):
Receiving an indication associated to events associated to the additional objects
For each event, receiving data associated;
Analyzing and generating a description of the object based on events.
No additional limitations are present.
With respect to claim 21, the claim(s) recite:
21. (New) The non-transitory computer readable medium of claim 1,
wherein the image data is a video of a basketball game, the first physical object is a first basketball player, the second physical object is a second basketball player, the first group of two or more events includes shots of a basketball to a hoop by the first basketball player, the second group of two or more events includes shots of the basketball to the hoop by the second basketball player, and the operations further comprise:
determining, based on an analysis of the video, a number of points scored by the first basketball player and a number of points scored by the second basketball player;
basing the determination to include in the textual content the description based on the first group of two or more events of the first physical object on the number of points scored by the first basketball player; and
basing the determination not to include in the textual content any description based on the second group of two or more events of the second physical object on the number of points scored by the second basketball player.
This reads on a human (e.g., mentally and/or using pen and paper):
Wherein the received printed image is associated to a basketball game, the first object being a first player, the second object being a second player and the group of events comprise details about the shots of a basketball to a hoop by the first and the second players;
Determining or counting the number of points by first player and by second player;
Writing down information associated with the points/scores for the first player;
Not including points/scores associated with the second player.
No additional limitations are present.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-5, 7, 9-10, 15-16, and 18-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhang et al. (US 20190366153 A1).
As to independent claim 1, Zhang et al. teaches:
1. A non-transitory computer readable medium storing computer implementable instructions that when executed by at least one processor cause the at least one processor to perform operations for generating a textual content selectively reporting objects (see ¶ [0024]: “In another aspect, one embodiment of the present invention is a system for generating ball shot analytics using a single mobile computing device, comprising at least one processor on the mobile computing device, and a non-transitory physical medium for storing program code and accessible by the processor, the program code when executed by the processor causes the processor to receive an input video of a ball game, and a location of a shooter in a shooter identification frame of the input video, detect one or more balls and player postures from the input video, generate one or more ball flows and one or more posture flows by grouping the detected balls and the detected player postures along a time line, identify a generated player posture flow as a shooter posture flow, based on the input location of the shooter, identify a generated ball flow as related to the shooter posture flow, determine a ball-from-shooter time by backtracking the related ball flow from a shot attempt, determine a shot event occurring before the ball-from-shooter time, and generate one or more shot analytics based on the shot event, the shooter posture flow, and the related ball flow.”), the operations comprising:
receiving image data (see ¶ [0009]: “More specifically, in one aspect, one embodiment of the present invention is a method for generating ball shot analytics using a single mobile computing device, comprising the steps of receiving an input video of a ball game and a location of a shooter in a shooter identification frame of the input video, detecting one or more balls and player postures from the input video, generating one or more ball flows and one or more posture flows by grouping the detected balls and the detected player postures along a time line, identifying a generated player posture flow as a shooter posture flow, based on the input location of the shooter, identifying a generated ball flow as related to the shooter posture flow, determining a ball-from-shooter time by backtracking the related ball flow from a shot attempt, determining a shot event occurring before the ball-from-shooter time, and generating one or more shot analytics based on the shot event, the shooter posture flow, and the related ball flow.” and ¶ [0070]: “…A computing device 110 may comprise at least one camera for capturing various image and video footage 120 of game actions, and may implement a NEX system 150 for generating shot analytics 190 such as shot type and back angle 192, leg power 194, and shot release statics 196...”);
calculating a convolution of at least part of the image data to thereby obtain a result value of the calculated convolution of the at least part-of the image data (see ¶ [0085]: “In process step 210, objects of interests are detected from frames of the input video. In particular, one or more convolutional neural networks (CNN) may be applied to identify desired objects including balls and players in the input video, and the detected objects are passed as input 215 to process step 220. Each CNN module may be trained using one or more prior input videos. In individual training sessions, only a single player is present, although multiple balls may be moving through the court if a basketball shooting machine is used. In multiple-player training sessions or games, multiple players and multiple balls may be present. A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”);
identifying a plurality of physical objects based on the result value of the calculated convolution of the at least part of the image data, the plurality of physical objects includes at least a first physical object and a second physical object (see Fig. 9B (players: 904-905, ball: 902, and basket/hoop: 980), ¶ [0085] citation as in limitation above. More specifically: “…A CNN utilizes the process of convolution to capture the spatial and temporal dependencies in an image, and to extract features from the input video for object detection. Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures. A ball moves through space, leading to changing size and location from video frame to video frame. A player also moves through space while handling the ball leading to both changing locations, sizes, and body postures.”, and ¶ [0154]: “Embodiments of the present invention may first detect the ball, determine a corresponding trajectory, then trace the ball trajectory to see if it ends in a shot attempt. For example, a box 902 in FIG. 9A represents a ball extraction result with confidence value of 1.000. Trajectory 903, represented as a dotted curve in FIG. 9A, may be reconstructed directly from a ball flow comprising a sequence of ball objects, or be generated by interpolating and/or extrapolating several known ball positions in air. Trajectory 903 represents a pass from player 905 to player 904, where the ball does not move above any of these two player's upper bodies, or come close to basket 980. By comparison, trajectory 963 in FIG. 9G corresponds to a shot attempt by shooter 904. Once a ball flow or trajectory such as 963 is determined, the ball flow can be examined to determine whether the ball has been thrown from the shooter's upper body upward, and if so, declare it as a shot attempt.”
Here, the first physical object is analogous to player 904, while the second physical object is analogous to player 905.);
receiving an indication of a first group of two or more events in the physical world caused by the first physical object (see ¶ [0020 and 0099-0100]: “[0020] In some embodiments, the shot event is selected from the group consisting of dribble event, jump event, catch-ball event, ball-leave-hand event, one-two leg jump, shooter's foot-on-ground movement, and the shot type is selected from the group consisting of layup, regular shot, dribble-pull-up, off-the-move, and catch-and-shoot. [0073] FIG. 1B is a flow diagram 190 providing a process overview of using a mobile device-based NEX system 150 to generate shot analytics and statistics, according to one embodiment of the present invention. This exemplary process takes as inputs a video segment or video stream, and/or a shooter's location in any given frame of the video input. Through new and novel methods for computer vision and algorithmic analysis, systems and devices implemented according to embodiments of the present invention extract various shot analytics, including, but are not limited to, shot type, release time, release angle, shooter body bend angle, leg bend ratio, moving speed and direction, and height of a jump event. The input video may be a live-stream, or an off-line recording, and may be a single perspective video, also known as a monocular video. [0099] With filtered flow and shot information 315, the NEX system may apply the remaining process steps in FIG. 3 to determine one or more shot events occurring before the ball-from-shooter time, and to generate one or more shot analytics 185 based on the one or more shot events, the shooter posture flow, and the related ball flow. In this disclosure, a “shot event” refers to player actions leading up to a shot attempt. That is, a shot event describes player movements before the ball leaves the shooter's hand in a shot attempt. A shot event may occur right before a shot is launched, or some time shortly before the shot is launched. [0100] In process step 320 shown in FIG. 3, several exemplary shot events are detected, for example, a dribble event, a jump event, a catch-ball event, as well as shooter movement in image space. Detected shot events, shooter movement in image space, shooter posture flow, and ball-shoot-from-hand time are used as input 325 to further processing steps 329, 330 and optionally 331, to determine one or more shot analytics.”
Here, the Examiner notes that the indication of a first group of two or more events is read by disclosures in Zhang et al. regarding events associated with player 904 and the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.).);
receiving an indication of a second group of two or more events in the physical world caused by the second physical object (see ¶ [0020 and 0099-0100] citations as in limitation above and further ¶ [0150]: “In table 810, raw information is divided into ball information 812, shooter information 814, events information 816, and scene information 818. For balls extracted from the input video, one or more ball flow and trajectories may be identified, and shot attempts may be determined based on the ball trajectories and their positions relative to the hoop. For the shooter, pose information may be determined from, for example, 18 key points on the body. Following a shot attempt trajectory, shooter poses may be detected in the region around the ball, and tracked as shooter poses. In some embodiments, more than one player may be present, and shooter information 814 may refer to player pose information and player posture flow as discussed with reference to FIGS. 1B to 4. In addition, shooter information 814 may be correlated with ball information 812 to determine different shot events such as ball-leave-hand, jump, dribble, and catch-ball events. Scene information 818 includes how hoop, court, and other relevant objects of interests are placed within the image domain, including hoop detection information and how court is placed in the image. Such scene information may be combined with other ball, shooter, and events information to generate shot analytics and/or game analytics, such as determining whether a shot is a 3-pointer or not.”
Here, the Examiner notes that the indication of a second group of two or more events is read by disclosures in Zhang et al. regarding events associated with the second player 905 and the ball OR by disclosures of other scene information (e.g., hoop, court or other relevant objects present in the image).);
based on the first group of two or more events, determining to include in a textual content a description based on the first group of two or more events of the first physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the determining to include in textual content a description based on the first group of two or more events is read by disclosures in Zhang et al. regarding shot analytics or the events associated with player 904 managing the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
based on the second group of two or more events, determining not to include in the textual content any description based on the second group of two or more events of the second physical object (see ¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the Examiner notes that no information regarding the additional player 905 OR scene information (e.g., hoop) are included in the shot analytics as disclosed in Zhang et al., wherein only shot analytics or the events associated with player 904 managing the ball are included (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.);
for each event of the first group of one or more events, receiving data associated with the event (see ¶ [0020-0021, 0099-0100, and 0150] citations as in limitation(s) above. More specifically: ¶ [0099] With filtered flow and shot information 315, the NEX system may apply the remaining process steps in FIG. 3 to determine one or more shot events occurring before the ball-from-shooter time, and to generate one or more shot analytics 185 based on the one or more shot events, the shooter posture flow, and the related ball flow. In this disclosure, a “shot event” refers to player actions leading up to a shot attempt. That is, a shot event describes player movements before the ball leaves the shooter's hand in a shot attempt. A shot event may occur right before a shot is launched, or some time shortly before the shot is launched. [0100] In process step 320 shown in FIG. 3, several exemplary shot events are detected, for example, a dribble event, a jump event, a catch-ball event, as well as shooter movement in image space. Detected shot events, shooter movement in image space, shooter posture flow, and ball-shoot-from-hand time are used as input 325 to further processing steps 329, 330 and optionally 331, to determine one or more shot analytics.”);
analyzing the data associated with the first group of two or more events to generate a particular description of the first physical object (see ¶ [0020-0021, 0099-0100, and 0150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event). More specifically: ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.” and further ¶ [0066]: “To analyze shot attempts, embodiments of the present invention take as an input a shot attempt video, and/or a shooter's image location at any given frame in the video. The input video may be a real-time video stream from a live-camera, or a recorded video. Computer vision techniques such as a convolutional neural network (CNN) may then be applied to some or all frames of the shot attempt video to detect the basketballs, individual players and their postures in the video, close to the supplied shooter. A tracking algorithm may be performed to track all detected balls and postures, where multiple balls or postures may be present in each frame of the shot attempt video, leading to multiple ball flows and posture flows. An object flow consists of object instances from different video frames, and can be viewed as a time-sequence of object positions as traversed by the object. All object instances in the same flow are considered the same object. For example, all instances of a ball having changing spatial locations in some consecutive frames of the video are identified as the same ball and viewed as a ball flow; all instances of a player having changing postures and possibly changing spatial locations in some consecutive frames of the video are identified as the same player, and viewed as a player posture flow…”);
generating the textual content, the textual content includes the particular description of the first physical object and does not include any description based on the second group of two or more events of the second physical object (see ¶ [0020, 0099-0100, and 0150] citations as in limitation(s) above. More specifically: Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the generating the textual content including description of the first physical object (e.g., player 904 managing the ball) and not of the second physical object is read by disclosures in Zhang et al. regarding shot analytics or the events associated with the player 904 managing the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”. Here, it is noted that no description associated with other elements or objects like the additional player 905 or hoop (i.e., second physical object) are included in the shot analytics.); and
providing the generated textual content (see ¶ [0020, 0099-0100, and 0150] citations as in limitation(s) above. More specifically: Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event))).
As to independent claim 19, Zhang et al. teaches:
19. A system for generating a textual content selectively reporting objects (see ¶ [0024]: “In another aspect, one embodiment of the present invention is a system for generating ball shot analytics using a single mobile computing device, comprising at least one processor on the mobile computing device, and a non-transitory physical medium for storing program code and accessible by the processor, the program code when executed by the processor causes the processor to receive an input video of a ball game, and a location of a shooter in a shooter identification frame of the input video, detect one or more balls and player postures from the input video, generate one or more ball flows and one or more posture flows by grouping the detected balls and the detected player postures along a time line, identify a generated player posture flow as a shooter posture flow, based on the input location of the shooter, identify a generated ball flow as related to the shooter posture flow, determine a ball-from-shooter time by backtracking the related ball flow from a shot attempt, determine a shot event occurring before the ball-from-shooter time, and generate one or more shot analytics based on the shot event, the shooter posture flow, and the related ball flow.”), the system comprising:
at least one processing unit configured to perform the operations (see ¶ [0024] citation as in preamble above. More specifically: “…comprising at least one processor on the mobile computing device…”) of:
[the limitations as in claim 1, above].
As to independent claim 20, Zhang et al. teaches:
20. A method for generating a textual content selectively reporting objects (see ¶ [0024] citation as in claim 19, above and further ¶ [0003]: “Embodiments of the present invention are in the field of sports analysis, and pertain particularly to methods and systems for generating real-time statistical analytics of sports and related games with a mobile device having a camera for video capturing.”), the method comprising:
[the limitations as in claim 1, above].
Regarding claim 2, Zhang et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
2. The non-transitory computer readable medium of claim 1,
wherein
the determination to include in the textual content the description based on the first group of one or more events of the first physical object is further based on the second group of one or more events (see ¶ ¶ [0154]: “Embodiments of the present invention may first detect the ball, determine a corresponding trajectory, then trace the ball trajectory to see if it ends in a shot attempt. For example, a box 902 in FIG. 9A represents a ball extraction result with confidence value of 1.000. Trajectory 903, represented as a dotted curve in FIG. 9A, may be reconstructed directly from a ball flow comprising a sequence of ball objects, or be generated by interpolating and/or extrapolating several known ball positions in air. Trajectory 903 represents a pass from player 905 to player 904, where the ball does not move above any of these two player's upper bodies, or come close to basket 980. By comparison, trajectory 963 in FIG. 9G corresponds to a shot attempt by shooter 904. Once a ball flow or trajectory such as 963 is determined, the ball flow can be examined to determine whether the ball has been thrown from the shooter's upper body upward, and if so, declare it as a shot attempt.”), and
the determination not to include in the textual content any description based on the second group of one or more events of the second physical object is further based on the first group of one or more events (see ¶ [0154] citation as in limitation above.
Here, the determination of a shot attempt is what essentially determines what player to focus on for the shot analytics as seen from example ¶ [0154] player 904 receives the ball from the additional player 905, and shot analytics are associated to the shot attempt by player or shooter 904.).
Regarding claim 3, Zhang et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
3. The non-transitory computer readable medium of claim 1,
wherein
the plurality of physical objects further includes a third object (see Figs. 19A-B and 21-22 and ¶ [0165-0166]: “[0165] FIGS. 19A and 19B are respective screen captures 1900 and 1940 from a third exemplary input shot attempt video and illustrate a catch-and-shoot shot, according to one embodiment of the present invention. In this illustrative example, three players 1910, 1920, and 1930 are present, as well as two balls 1901 and 1902. The NEX system detects individual balls and player postures respectively. Similarly, FIG. 20 is a screen capture 2000 from a fourth exemplary input shot attempt video and illustrates an on-the-move shot. FIG. 21 a screen capture 2100 rom a fifth exemplary input shot attempt video and illustrates an off-the-dribble shot.”)),
the third physical object is associated with a third group of two or more events (see Figs. 19A-B and 21-22 and ¶ [0165] citations as in limitation above and further ¶ [0166]: “[0166] FIGS. 22, 23 and 24 are screen captures 2200, 2300, and 2400 from three other exemplary input shot attempt videos respectively and illustrate back angle statistics throughout the video as a function of the frame number, according to some embodiments of the present invention. In FIG. 22, back angle of player 2204, as computed from player posture 2206, is plotted against time in sub-figure 2250, with the shooter back angle in the current frame marked as data point 2255. In FIG. 23, back angle of player 2304, as computed from player posture 2306, is plotted against time in sub-figure 2350, with the shooter back angle in the current frame marked as data point 2355. Similarly in FIG. 24, back angle of player 2404, as computed from player posture 2406, is plotted against time in sub-figure 2450, with the shooter back angle in the current frame marked as data point 2455. As player 2404 has his back almost straight up, the measured back angle 2455 is close to 0 degrees.”),
the third group of two or more events includes at least one event not included in the first group and not included in the second group (see Figs. 19A-B and 21-22 and ¶ [0165] citations as in limitation above. More specifically: Fig. 19A-B: third player’s (1930) move associated with the ball are itself a group of events independent from the other two players (1910 and 1920).),
the determination to include in the textual content the description based on the first group of two or more events of the first physical object is further based on the third group of two or more events (see Figs. 19A-B and 21-22 and ¶ [0165] citations as in limitation above. More specifically: Fig. 19A-B: third player’s (1920) move associated with the ball are itself a group of events (e.g., first group of events) at least indirectly based on the third group of events because the player catches the ball that has been thrown at him/her to shoot (i.e., as can be seen in the shooting info: “shooting type: catch and shoot”).), and
the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is further based on the third group of one or more events (see ¶ [0154] citation as in limitation above.
Here, the determination of a shot attempt is what essentially determines what player to focus on for the shot analytics as seen from example ¶ [0154] player 904 receives the ball from the additional player 905, and shot analytics are associated to the shot attempt by player or shooter 904. In the case of Figs. 19A-B and 21-22, a similar situation occurs in which the determination of a shot attempt is what essentially determines what player to focus on for the shot analytics, in which player 1920 receives the ball from the additional player 1910, and shot analytics are associated to the shot attempt by player or shooter 1920 as seen in Figs. 19A-B.).
Regarding claim 4, Zhang et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
4. The non-transitory computer readable medium of claim 1,
wherein
the determination to include in the textual content the description based on the first group of one or more events of the first physical object is further based on a type of the first physical object and a type of the second physical object (see Figs. 19A-B and 21-22 and ¶ [0165] citations as in limitation above. More specifically: Fig. 19A-B: third player’s (1920) move associated with the ball are itself a group of events (e.g., first group of events) at least indirectly based on the third group of events because the player catches the ball that has been thrown at him/her to shoot (i.e., as can be seen in the shooting info: “shooting type: catch and shoot”). and further see ¶ [0124]: “In process step 340, different shot types or shooting types may be identified. Inputs to this process include one or more of previously detected shot events, previously determined shot analytics, and optionally movement in court bird-eye. Shot types include, but are not limited to, catch-and-shoot, dribble-pull-up, and layup shots. Shot types may be viewed as a category of qualitative shot analytics.” and ¶ [0151]: “…Examples of shot types 866 include, but are not limited to, layup, catch-and-shoot, on-the-move, off-the-dribble, regular, gloater/runner/hook, and stepback/jab. Derivation of such movement information may require events information and movement information from table 810.”
Here, the Examiner notes that each player in Zhang et al. as seen in Figs. 11A,19A-B, 21-22 are associated with a type of shot, which reads on the type of first physical object and the type of second physical object.), and
the determination not to include in the textual content any description based on the second group of one or more events of the second physical object is further based on the type of the first physical object and the type of the second physical object (see ¶ [0154] citation as in limitation above.
Here, the determination of a shot attempt is what essentially determines what player to focus on for the shot analytics as seen from example ¶ [0154] player 904 receives the ball from the additional player 905, and shot analytics are associated to the shot attempt by player or shooter 904. In the case of Figs. 19A-B and 21-22, a similar situation occurs in which the determination of a shot attempt is what essentially determines what player to focus on for the shot analytics, in which player 1920 receives the ball from the additional player 1910, and shot analytics are associated to the shot attempt by player or shooter 1920 as seen in Figs. 19A-B. Also, the Examiner notes that each player in Zhang et al. as seen in Figs. 11A,19A-B, 21-22 are associated with a type of shot, which reads on the type of first physical object and the type of second physical object.).
Regarding claim 5, Zhang et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
5. The non-transitory computer readable medium of claim 1,
wherein
the determination to include in the textual content the description based on the first group of two or more events of the first physical object is further based on a type associated with a specific event of the first group of one or more events (see Figs. 19A-B and 21-22 and ¶ [0165] citations as in limitation above. More specifically: Fig. 19A-B: third player’s (1920) move associated with the ball are itself a group of events (e.g., first group of events) at least indirectly based on the third group of events because the player catches the ball that has been thrown at him/her to shoot (i.e., as can be seen in the shooting info: “shooting type: catch and shoot”). and further see ¶ [0124]: “In process step 340, different shot types or shooting types may be identified. Inputs to this process include one or more of previously detected shot events, previously determined shot analytics, and optionally movement in court bird-eye. Shot types include, but are not limited to, catch-and-shoot, dribble-pull-up, and layup shots. Shot types may be viewed as a category of qualitative shot analytics.” and ¶ [0151]: “…Examples of shot types 866 include, but are not limited to, layup, catch-and-shoot, on-the-move, off-the-dribble, regular, gloater/runner/hook, and stepback/jab. Derivation of such movement information may require events information and movement information from table 810.”
Here, the Examiner notes that each player in Zhang et al. as seen in Figs. 11A,19A-B, 21-22 are associated with a type of shot, which reads on the type of first physical object and the type of second physical object. Also, the specific event would be read by the specific type of shots from one player to the other (e.g., catch and shoot).), and
the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is further based on a type associated with a specific event of the second group of one or more events (see ¶ [0154] citation as in limitation above.
Here, the determination of a shot attempt is what essentially determines what player to focus on for the shot analytics as seen from example ¶ [0154] player 904 receives the ball from the additional player 905, and shot analytics are associated to the shot attempt by player or shooter 904. In the case of Figs. 19A-B and 21-22, a similar situation occurs in which the determination of a shot attempt is what essentially determines what player to focus on for the shot analytics, in which player 1920 receives the ball from the additional player 1910, and shot analytics are associated to the shot attempt by player or shooter 1920 as seen in Figs. 19A-B. Also, the Examiner notes that each player in Zhang et al. as seen in Figs. 11A,19A-B, 21-22 are associated with a type of shot, which reads on the type of first physical object and the type of second physical object and that the specific event would be read by the specific type of shots from one player to the other (e.g., pass for the catch and shoot).).
Regarding claim 7, Zhang et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
7. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
analyzing the image data to detect events of the first group of two or more events (¶ [0020, 0099-0100, and 150] citations as in limitation(s) above and further Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the determining to include in textual content a description based on the first group of two or more events is read by disclosures in Zhang et al. regarding shot analytics or the events associated with player 904 managing the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”.); and
analyzing the image data to detect events of the second group of two or more events (see ¶ [0020, 0099-0100, and 0150] citations as in limitation(s) above. More specifically: Fig. 11A (1142 (shooting info) and 1115 (jump event and ball-leave hand event)) and ¶ [0021]: “In some embodiments, the shot analytics is selected from the group consisting of release time, back angle, leg bend ratio, leg power, moving speed, moving direction, and height of jump.”
Here, the generating the textual content including description of the first physical object (e.g., player 904 managing the ball) and not of the second physical object is read by disclosures in Zhang et al. regarding shot analytics or the events associated with the player 904 managing the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). For example, as seen in Fig. 11A: “Shooting info”, “jump event, ball-leave-hand event”. Here, it is noted that no description associated with other elements or objects like the additional player 905 or hoop (i.e., second physical object) are included in the shot analytics.).
Regarding claim 9, Zhang et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
9. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
determining a first magnitude associated with the first group of two or more events (see Fig. 11A: (shooting info (e.g., release time: 0.90 s, back-angle: 0-42, leg power: 0.706 HIGH)) and ¶ [0020 and 0099-0100]: “[0020] In some embodiments, the shot event is selected from the group consisting of dribble event, jump event, catch-ball event, ball-leave-hand event, one-two leg jump, shooter's foot-on-ground movement, and the shot type is selected from the group consisting of layup, regular shot, dribble-pull-up, off-the-move, and catch-and-shoot. [0073] FIG. 1B is a flow diagram 190 providing a process overview of using a mobile device-based NEX system 150 to generate shot analytics and statistics, according to one embodiment of the present invention. This exemplary process takes as inputs a video segment or video stream, and/or a shooter's location in any given frame of the video input. Through new and novel methods for computer vision and algorithmic analysis, systems and devices implemented according to embodiments of the present invention extract various shot analytics, including, but are not limited to, shot type, release time, release angle, shooter body bend angle, leg bend ratio, moving speed and direction, and height of a jump event. The input video may be a live-stream, or an off-line recording, and may be a single perspective video, also known as a monocular video. [0099] With filtered flow and shot information 315, the NEX system may apply the remaining process steps in FIG. 3 to determine one or more shot events occurring before the ball-from-shooter time, and to generate one or more shot analytics 185 based on the one or more shot events, the shooter posture flow, and the related ball flow. In this disclosure, a “shot event” refers to player actions leading up to a shot attempt. That is, a shot event describes player movements before the ball leaves the shooter's hand in a shot attempt. A shot event may occur right before a shot is launched, or some time shortly before the shot is launched. [0100] In process step 320 shown in FIG. 3, several exemplary shot events are detected, for example, a dribble event, a jump event, a catch-ball event, as well as shooter movement in image space. Detected shot events, shooter movement in image space, shooter posture flow, and ball-shoot-from-hand time are used as input 325 to further processing steps 329, 330 and optionally 331, to determine one or more shot analytics.”
Here, the Examiner notes that the indication of a first group of two or more events is read by disclosures in Zhang et al. regarding events associated with player 904 and the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). Also, the Examiner notes that the magnitude of the first group of two or more events is read by shooting info values/magnitudes corresponding to a first player/shooter (e.g., release time: 0.90 s, back-angle: 0-42, leg power: 0.706 HIGH).);
based on the first magnitude, determining to include in the textual content the description based on the first group of two or more events of the first physical object (see Fig. 11A: (shooting info (e.g., release time: 0.90 s, back-angle: 0-42, leg power: 0.706 HIGH)) and ¶ [0020, 0073 and 0099-0100] citations as in limitation above.
Here, the Examiner notes that the indication of a first group of two or more events is read by disclosures in Zhang et al. regarding events associated with player 904 and the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.). Also, the Examiner notes that the magnitude of the first group of two or more events is read by shooting info values/magnitudes corresponding to a first player/shooter (e.g., release time: 0.90 s, back-angle: 0-42, leg power: 0.706 HIGH) which are displayed as textual content as part of the shot analytics.);
determining a second magnitude associated with the second group of two or more events (see Fig. 11A: (shooting info (e.g., release time: 0.90 s, back-angle: 0-42, leg power: 0.706 HIGH)) and ¶ [0020, 0073 and 0099-0100] citations as in limitation above and further ¶ [0118]: “To detect shooter movement in an image, the shooter's movement, or movement of the shooter's foot in the image space may be computed by examining the shooter's foot location as obtained from his or her posture. To compensate for posture inaccuracy or occlusion, each frame's shooter location or foot location may be smoothed as a weighted or unweighted average or median over a sliding time window. For example, similarity of posture sizes may be used as weights for the smoothing process, and x and y values of the locations may be smoothed separately.”
Here, the Examiner notes that magnitudes (e.g., position) associated with the second or additional shooter or player managing (or not) the ball will read on the second magnitude).); and
based on the second magnitude, determining not to include in the textual content any description based on the second group of two or more events of the second physical object (see Fig. 11A: (shooting info (e.g., release time: 0.90 s, back-angle: 0-42, leg power: 0.706 HIGH)) and ¶ [0020, 0073 and 0099-0100] citations as in limitation above and further ¶ [0118]: “To detect shooter movement in an image, the shooter's movement, or movement of the shooter's foot in the image space may be computed by examining the shooter's foot location as obtained from his or her posture. To compensate for posture inaccuracy or occlusion, each frame's shooter location or foot location may be smoothed as a weighted or unweighted average or median over a sliding time window. For example, similarity of posture sizes may be used as weights for the smoothing process, and x and y values of the locations may be smoothed separately.”
Here, the Examiner notes that magnitudes (e.g., position) associated with the second or additional shooter or player managing (or not) the ball will read on the second magnitude). Also, that this information is used to determine the actual shooter or player information to be generated or displayed as seen in Fig. 11A.).
Regarding claim 10, Zhang et al. teaches the limitations as in claim 9, above.
Zhang et al. further teaches:
10. The non-transitory computer readable medium of claim 9,
wherein
the determination to include in the textual content the description based on the first group of two or more events of the first physical object is further based on the second magnitude (see Fig. 11A: (shooting info (e.g., release time: 0.90 s, back-angle: 0-42, leg power: 0.706 HIGH)) and ¶ [0020, 0073, 0099-0100 and 0108] citations as in limitation above.
Here, the Examiner notes that magnitudes (e.g., position) associated with the second or additional shooter or player managing (or not) the ball will read on the second magnitude). Also, that this information is used to determine the actual shooter or player information to be generated or displayed as seen in Fig. 11A.), and
wherein the determining not to include in the textual content any description based on the second group of one or more events of the second physical object is further based on first magnitude (see Fig. 11A: (shooting info (e.g., release time: 0.90 s, back-angle: 0-42, leg power: 0.706 HIGH)) and ¶ [0020, 0073, 0099-0100 and 0108] citations as in limitation above.
Here, the Examiner notes that magnitudes (e.g., position) associated with the first and second/additional shooter or player managing (or not) the ball will read on the first and second magnitudes). Also, that this information is used to determine the actual shooter or player information to be generated or displayed as seen in Fig. 11A.).
Regarding claim 15, Zhang et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
15. The non-transitory computer readable medium of claim 1,
wherein the textual content is associated with a topic (see ¶ [0063, 0119, 0150, and 0157]: “[0063] More specifically, embodiments of the present invention relate to tracking a shot attempt and the corresponding player's motion, form, or posture throughout the shot attempt, in the forward and/or backward direction, and providing analytics relevant to the shot attempt, all by a mobile computing device such as a smartphone. [0119] In process step 329, a release time is determined, based on shot events generated in step 320, such as a catch-ball event and one or more dribble events. For example, the release time may be computed by choosing the latter of a catch-ball event and the last dribble event, and by calculating the time between this chosen event and the ball-shoot-from-hand time. If neither catch-ball event nor dribble event exists in the given time period such as 2 seconds, release hand time may be too long to be relevant, thus does not need to be computed. [0150] …In addition, shooter information 814 may be correlated with ball information 812 to determine different shot events such as ball-leave-hand, jump, dribble, and catch-ball events. Scene information 818 includes how hoop, court, and other relevant objects of interests are placed within the image domain, including hoop detection information and how court is placed in the image. Such scene information may be combined with other ball, shooter, and events information to generate shot analytics and/or game analytics, such as determining whether a shot is a 3-pointer or not. [0157] A set 1115 of six time bars are provided in FIGS. 11A to 11D by the NEX system to indicate relevant shot events. From top to bottom, they respectively represent a catch-ball event, raw signal captured for catch-ball-event, dribble event, raw signal captured for dribble event, jump event, and ball-leave-hand event. Each shaded block spans the duration of a detected shot event. Confidence values for event estimation are provided to the right for each event type, changing on a per-frame basis.”),
the determination to include in the textual content the description based on the first group of two or more events of the first physical object is based on a degree of relevance of the first physical object to the topic (see ¶ [0063, 0119, 0150, and 0157] citations as in limitation above.
Here, the Examiner notes that the provided information regarding the shot events, including ball and/or hoop detection information (e.g., confidence value) reads on the determination to include in the textual content events based on the degree of relevance.), and
the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is based on a degree of relevance of the second see ¶ [0063, 0119, 0150, and 0157] citations as in limitation above.
Here, the Examiner notes that the provided information regarding the shot events, including ball and/or hoop detection information (e.g., confidence value) reads on the determination to include in the textual content events based on the degree of relevance.)object to the topic (see ¶ [0063, 0119, 0150, and 0157] citations as in limitation above.
Here, the Examiner notes that the provided information regarding the shot events, including ball and/or hoop detection information (e.g., confidence value) also reads on the determination on what not to include in the textual content events based on the degree of relevance.)
Regarding claim 16, Zhang et al. teaches the limitations as in claim 1, above.
Zhang et al. further teaches:
16. The non-transitory computer readable medium of claim 1,
wherein the plurality of physical objects includes at least two additional physical objects in addition to the first and second physical objects (see Fig. 9A and 17 (Ball box and confidence value presented in Fig. 9A and hoop box inf Fig. 17) and ¶ [0154 and 0163]: “[0154] Embodiments of the present invention may first detect the ball, determine a corresponding trajectory, then trace the ball trajectory to see if it ends in a shot attempt. For example, a box 902 in FIG. 9A represents a ball extraction result with confidence value of 1.000. [0163] FIG. 17 shows a screen capture 1700 from the input shot attempt video in FIGS. 9A to 9G and illustrates various scene and movement indicators, according to one embodiment of the present invention. In particular, a hoop 1780 is detected and outlined with solid line 1785, while court lines 1720 are detected and traced with dotted lines. Position 1702 of the shooter and shooter foot location trajectory 1712 are shown in a bird-eye-view 1790 of the court. Additional shooter movement information such as instantaneous moving speed and jump height may also be computed using the methods and systems as disclosed herein.”), and the operations further comprise:
selecting a subset of at least one but not all of the at least two additional object (see Fig. 9A and 17 (Ball box and confidence value presented in Fig. 9A and hoop box inf Fig. 17) and ¶ [0154 and 0163] citations as in limitation above. More specifically: ball box/enclosure 902);
for each object in the subset, generating a description of the object (see Fig. 9A and 17 (Ball box and confidence value presented in Fig. 9A and hoop box inf Fig. 17) and ¶ [0154 and 0163] citations as in limitation above. More specifically: ball box/enclosure 902 and the confidence value); and
including the generated descriptions of all objects in the subset in the generated textual content (see Fig. 9A and 17 (Ball box and confidence value presented in Fig. 9A and hoop box inf Fig. 17) and ¶ [0154 and 0163] citations as in limitation above. More specifically: ball box/enclosure 902 and the confidence value).
Regarding claim 17, Zhang et al. teaches the limitations as in claim 16, above.
Zhang et al. further teaches:
17. The non-transitory computer readable medium of claim 16,
wherein the operations further comprise, for each physical object of the at least two additional object (see Fig. 17 (ball detection box/enclosure 902 and hoop detection box/enclosure 1780) and ¶ [0155]: “…For example, a “shot attempt” ball trajectory may be followed, and all postures in regions around the ball may be detected, where each posture may be presented by a predetermined number of key points. A detected posture or posture flow may be determined to represents a shooter, if the related ball trajectory is found to represent a shot attempt.”); and
based on the group of two or more events associated with the physical object, determining whether to include the physical object in the subset (see Fig. 17 (ball detection box/enclosure 902 and hoop detection box/enclosure 1780) and ¶ [0155] citation as in limitation above.).
Regarding claim 18, Zhang et al. teaches the limitations as in claim 16, above.
Zhang et al. further teaches:
18. The non-transitory computer readable medium of claim 16, wherein the operations further comprise,
for each object of the subset (see Fig. 17 (ball detection box/enclosure 902 and hoop detection box/enclosure 1780) and ¶ [0155] citation as in limitation above.):
receiving an indication of a group of two or more events associated with the object (see Fig. 17 (ball detection box/enclosure 902 and hoop detection box/enclosure 1780) and ¶ [0155] citation as in limitation above.);
for each event of the group of two or more events associated with the object, receiving data associated with the event (see Fig. 17 (ball detection box/enclosure 902 and hoop detection box/enclosure 1780, trajector(ies) 1790) and ¶ [0155] citation as in limitation above.); and
analyzing the data associated with the group of two or more events associated with the physical object to generate the description based on the group of two or more events associated with the physical object of the object (see Fig. 17 (ball detection box/enclosure 902 and hoop detection box/enclosure 1780, trajector(ies) 1790) and ¶ [0155] citation as in limitation above.), thereby generating the description of the object (see Fig. 17 (ball detection box/enclosure 902 and hoop detection box/enclosure 1780, trajector(ies) 1790) and ¶ [0155] citation as in limitation above.).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 20190366153 A1) as applied to claim 1 above, and further in view of Wu et al. (US 11336935 B1).
Regarding claim 6, Jolly et al. teaches the limitations as in claim 1, above.
However, Zhang et al. does not explicitly teach, but Wu et al. does teach:
6. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
receiving audio data captured from the environment using at least one audio sensor (see ¶ starting at Col. 1, line 30: “(3) This disclosure describes techniques for techniques for identifying audio-visual desynchronization and for synchronizing audio content and video content. A media content provider providing a given piece of media content may want to check the media content for audio-visual desynchronization, as consumers can be dissatisfied if an audio stream excessively leads or lags its associated video stream. The present disclosure provides a mechanism to identify audio-visual desynchronization. As an example, a media content provider can identify certain kinds of audio events (e.g., the sound of a tennis ball being hit by a racket, a tennis ball bouncing on a court, a basketball bouncing on a court or other structure, a baseball being hit by a bat, etc.) and can then analyze temporally-adjacent portions of the accompanying video stream for the presence of corresponding visual events (e.g., a tennis ball, basketball, or baseball rapidly changing directions across a small number of video frames)…” and ¶ starting at Col. 2, line 50: “(7) If the system detects an audio event (at 106), the system may then search for a corresponding video event (108). In some implementations, the system may search for a video event that occurs within a time window that includes the detected audio event. Additionally, the time window may be determined based on the limits of human perception and/or the limits of what consumers find to be acceptable desynchronization limits. In particular, humans are generally unable to perceive when audio lags the corresponding video by less than approximately 125 ms or when the audio leads the corresponding video by less than approximately 45 ms. Similarly, humans generally indicate a subjective acceptability of media content, at least for lip-sync errors, when the audio lags the video by less than approximately 200 ms or when the audio leads the video by less than approximately 100 ms…”);
accessing synchronization data configured to enable synchronization of the image data and the audio data (see ¶ starting at Col. 1, line 30 citation as in limitations above and further ¶ at Col. 3, lines 12-23: “… Limiting the amount of video that is searched for a video event (108) may also make it computationally easier to search for the video event, as the entirety of the video stream need not be searched. In such implementations and if the system is able to detect a corresponding video event (in 108) that occurs within or near the detectability and/or acceptability windows, relative to the detected audio event, the system can conclude that the video and audio streams are likely synchronized (112) and can provide a suitable notification that the streams are synchronized and/or continue with analyzing additional segments of the media content…” and ¶ starting at Col. 5, line 46: “(14) In some implementations, the system may search for video events in a window that is adjusted based on a previously-determined offset between audio and video streams. In particular, if the system detects a first offset between the audio and video streams at a first point along the media content timeline, the system may utilize that first offset in determining how far before and after the detected audio event to search for a video event. As an example, the system may search for a first video event within a temporal window that extends from about 80 ms before to about 240 ms after a first detected audio event along a media content timeline shared by the video and audio streams...”);
using the synchronization data to identify a first at least one portion of the audio data associated with the first physical object and to identify a second at least one portion of audio data associated with the second physical object (see ¶ starting at Col. 1, line 30, ¶ at Col. 3, lines 12-23 and ¶ starting at Col. 5, line 46 citations as in limitations above. More specifically, ¶ starting at Col. 1, line 30: “…As an example, a media content provider can identify certain kinds of audio events (e.g., the sound of a tennis ball being hit by a racket, a tennis ball bouncing on a court, a basketball bouncing on a court or other structure, a baseball being hit by a bat, etc.) and can then analyze temporally-adjacent portions of the accompanying video stream for the presence of corresponding visual events (e.g., a tennis ball, basketball, or baseball rapidly changing directions across a small number of video frames)…”
Here, the Examiner notes that the first portion associated with the first physical object is disclosed in Wu, as an example, by the tennis ball being hit by a racket, while the second portion would be the tennis ball bouncing on the court.);
analyzing the first at least one portion of the audio data to detect events of the first group of two or more events (see ¶ starting at Col. 1, line 30, ¶ at Col. 3, lines 12-23 and ¶ starting at Col. 5, line 46 citations as in limitations above. More specifically, ¶ starting at Col. 1, line 30: “…As an example, a media content provider can identify certain kinds of audio events (e.g., the sound of a tennis ball being hit by a racket, a tennis ball bouncing on a court, a basketball bouncing on a court or other structure, a baseball being hit by a bat, etc.) and can then analyze temporally-adjacent portions of the accompanying video stream for the presence of corresponding visual events (e.g., a tennis ball, basketball, or baseball rapidly changing directions across a small number of video frames)…”
Here, the Examiner notes that the first portion associated with the first physical object is disclosed in Wu, as an example, by the tennis ball being hit by a racket, while the second portion would be the tennis ball bouncing on the court, which are both analyzed by the system as disclosed in the citations above and Fig. 1.); and
analyzing the second at least one portion of the audio data to detect events of the second group of two or more events (see ¶ starting at Col. 1, line 30, ¶ at Col. 3, lines 12-23 and ¶ starting at Col. 5, line 46 citations as in limitations above.
Here, the Examiner notes that the first portion associated with the first physical object is disclosed in Wu, as an example, by the tennis ball being hit by a racket, while the second portion would be the tennis ball bouncing on the court, which are both analyzed by the system as disclosed in the citations above and Fig. 1.);.
Zhang et al. and Wu et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in event/object detection and/or analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. to incorporate the teachings of Wu et al. of receiving first modality data captured from an environment using at least one sensor of a first type of sensors; receiving second modality data captured from the environment using at least one sensor of a second type of sensors, the second type of sensors differs from the first type of sensors; analyzing the first modality data to detect the plurality of objects; accessing synchronization data configured to enable synchronization of the first modality data and the second modality data; using the synchronization data to identify a first at least one portion of the second modality data associated with the first object and to identify a second at least one portion of the second modality data associated with the second object; analyzing the first at least one portion of the second modality data to detect events of the first group of one or more events; and analyzing the second at least one portion of the second modality data to detect events of the second group of one or more events which provides the benefit of identifying audio-visual desynchronization and for synchronizing audio content and video content for consumer satisfaction (Col. 1, lines 31-35 of Wu et al.).
Claims 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 20190366153 A1) as applied to claim 1 above, and further in view of Pavithra et al. ("Spectral Clustering of Events in Social Media Flood Images based on Multimodal Analysis," 2021 7th International Conference on Electrical Energy Systems (ICEES), Chennai, India, 2021, pp. 648-653, (https://ieeexplore.ieee.org/abstract/document/93837277)).
Regarding claim 11, Zhang et al. teaches the limitations as in claim 1, above.
However, Zhang et al. does not explicitly teach, but Pavithra et al. does teach:
11. The non-transitory computer readable medium of claim 1, wherein the operations further comprise:
for each event of the first group of two or more events and for each event of the second group of two or more events, determining a mathematical object corresponding to the event in a mathematical space (see ¶ 1 of III. Proposed Work: “Spectral Clustering is one of the popular clustering algorithms. The Objective is to perform Event Clustering on Social media data using multimodal analysis that is both textual and visual image. The Proposed work Cluster the events by using both visual content, textual data and other parameters like temporal information and location. Highly related Microblogs based on Similarity constitute a Clique. Finally, Microblogs are clustered by using Spectral Clustering algorithm.” and ¶ 1-4 of III. Proposed Work: A. Text Analysis: “This module mainly focuses on analyzing the textual content. This gives the textual similarity by comparing the microblogs. Also make use of Multilevel Spatial and Temporal Information extraction to give temporal and spatial similarity. The first step is to preprocess the noisy text. Cleaning and preprocessing the noisy text are essential for any kind of analysis to be performed. This is done to have meaningful contents on which the techniques are applied. Preprocessing includes the following: lowercase transformation, Number removal, Punctuation removal, Whitespace removal, Stop word removal.
Textual Similarity: The next step is to extract the features using TDIDF Vectorizer. Using Cosine Similarity, the similarity between two tweets are computed. The Textual Cosine Similarity equation is given in equation 1. S(mi,mj) = <tci, tcj>/|tci| ࣭|tcj| (1) Here mi, mj are the microblogs and tci, tcj are the textual features.
Temporal Similarity: The date and time at which tweet is posted is recorded. While Extracting tweets from API, the timestamp information also extracted. The Temporal Similarity equation is given in equation 2. S(mi,mj) = 1 – [|ti - tj| / c ] (2) Here mi, mj are the microblogs, ti, tj are the timestamps of mi, mj respectively and c is a normalized constant.
Location Similarity: The next step is to compute Location Similarity. It follows a multilevel approach. First it checks for any location information in the Extracted dataset using API. If not then it checks in the tweet followed by the comments section. By which the location information is extracted at a multilevel approach. Then cosine similarity is used to compute the similarity. The Location Cosine Similarity between the microblogs mi, mj is given in equation 3. S(mi,mj) = <li, lj> / |li| ࣭|lj| (3)”
Here, the Examiner notes that the feature extractions (i.e.,Textual, Temporal, and Location similarities) in Pavithra et al. read on the determining multiple mathematical objects corresponding to events in a mathematical space (i.e., category).);
basing the determination to include in the textual content the description based on the first group of two or more events of the first physical object on the mathematical objects corresponding to the first group of two or more events (see Figure 2 (Similarity Graph Construction) and ¶ 1 of III. Proposed work: C. Microblog Clique Generations: “The next module is to generate the Microblog Clique by constructing the similarity matrix. Highly related posts constitute a Clique. The similarity matrix is constructed based on four parameters: Location Similarity, Textual Similarity, Visual Similarity and Temporal Similarity. Based on the Constructed Similarity matrix graph is constructed. Similarity value between two post is computed by averaging textual, visual, temporal and location similarity value. The rows and columns of the matrix are used to represent the nodes (Posts) in the graph. If the similarity value higher than or equal to threshold value, then two posts are highly related, else not. One post is compared with all other posts in the collection. Based on the computed similarity matrix, similarity graph is constructed. Similarity Graph is an undirected graph with Vertices and Edges. i.e. G= (V, E). Here Set of posts act as nodes V, Edge E exists if Similarity exists between two posts. Figure 2 shows the Similarity Graph constructed from Similarity matrix.”
Here, the Examiner notes that the feature extractions (i.e.,Textual, Temporal, and Location similarities) in Pavithra et al. read on the determining multiple mathematical objects corresponding to events in a mathematical space (i.e., category) and that this determination being based on highly related categories would read on the basing the determination to include textual content or not in a description.); and
basing the determination not to include in the textual content any description based on the second group of two or more events of the second physical object on the mathematical objects corresponding to the second group of two or more events (see Figure 2 (Similarity Graph Construction) and ¶ 1 of III. Proposed work: C. Microblog Clique Generations.
Here, the Examiner notes that the feature extractions (i.e.,Textual, Temporal, and Location similarities) in Pavithra et al. read on the determining multiple mathematical objects corresponding to events in a mathematical space (i.e., category) and that this determination being based on highly related categories would read on the basing the determination to include textual content or not in a description.).
Zhang et al. and Pavithra et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in event/object analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. to incorporate the teachings of Pavithra et al. of for each event of the first group of one or more events and for each event of the second group of one or more events, determining a mathematical object corresponding to the event in a mathematical space; basing the determination to include in the textual content the description based on the first group of one or more events of the first object on the mathematical objects corresponding to the first group of one or more events; and basing the determination not to include in the textual content any description based on the second group of one or more events of the second object on the mathematical objects corresponding to the second group of one or more events. which provides the benefit of improving performance of clustering algorithms helpful in the monitoring of events by public organizations (Abstract of Pavithra et al.).
Regarding claim 12, Zhang et al. teaches the limitations as in claim 11, above.
However, Zhang et al. does not explicitly teach, but Pavithra et al. does teach:
12. The non-transitory computer readable medium of claim 11, wherein the operations further comprise:
analyzing the mathematical objects corresponding to the first group of two or more events to determine a first mathematical object in the mathematical space, the first mathematical object differs from any mathematical object of the mathematical objects corresponding to the first group of two or more events (see ¶ 1 of III. Proposed Work and ¶ 1-4 of III. Proposed Work: A. Text Analysis citations as in claim 11, above.
Here, the Examiner notes that the feature extractions (i.e.,Textual, Temporal, and Location similarities) in Pavithra et al. read on the determining multiple mathematical objects corresponding to events in a mathematical space (i.e., category).);
analyzing the mathematical objects corresponding to the second group of two or more events to determine a second mathematical object in the mathematical space (see ¶ 1 of III. Proposed Work and ¶ 1-4 of III. Proposed Work: A. Text Analysis citations as in claim 11, above and further Figure 2 (Similarity Graph Construction) and ¶ 1 of III. Proposed work: C. Microblog Clique Generations: “The next module is to generate the Microblog Clique by constructing the similarity matrix. Highly related posts constitute a Clique. The similarity matrix is constructed based on four parameters: Location Similarity, Textual Similarity, Visual Similarity and Temporal Similarity. Based on the Constructed Similarity matrix graph is constructed. Similarity value between two post is computed by averaging textual, visual, temporal and location similarity value. The rows and columns of the matrix are used to represent the nodes (Posts) in the graph. If the similarity value higher than or equal to threshold value, then two posts are highly related, else not. One post is compared with all other posts in the collection. Based on the computed similarity matrix, similarity graph is constructed. Similarity Graph is an undirected graph with Vertices and Edges. i.e. G= (V, E). Here Set of posts act as nodes V, Edge E exists if Similarity exists between two posts. Figure 2 shows the Similarity Graph constructed from Similarity matrix.”
Here, the Examiner notes that the feature extractions (i.e.,Textual, Temporal, and Location similarities) in Pavithra et al. read on the determining multiple mathematical objects corresponding to events in a mathematical space (i.e., category) and that this determination being based on highly related categories would read on the basing the determination to include textual content or not in a description.), the second mathematical object differs from any mathematical object of the mathematical objects corresponding to the second group of two or more events (see ¶ 1 of III. Proposed Work and ¶ 1-4 of III. Proposed Work: A. Text Analysis citations as in claim 11, above and further Figure 2 (Similarity Graph Construction) and ¶ 1 of III. Proposed work: C. Microblog Clique Generations citations as in limitation above. More specifically: “… If the similarity value higher than or equal to threshold value, then two posts are highly related, else not. One post is compared with all other posts in the collection. Based on the computed similarity matrix, similarity graph is constructed…”);
basing the determination to include in the textual content the description based on the first group of two or more events of the first physical object on the first mathematical object (see ¶ 1 of III. Proposed Work and ¶ 1-4 of III. Proposed Work: A. Text Analysis citations as in claim 11, above and further Figure 2 (Similarity Graph Construction) and ¶ 1 of III. Proposed work: C. Microblog Clique Generations citations as in limitation above. More specifically: “… If the similarity value higher than or equal to threshold value, then two posts are highly related, else not. One post is compared with all other posts in the collection. Based on the computed similarity matrix, similarity graph is constructed…”
Here, the Examiner notes that the determination to include or not in textual content description based on events associated with mathematical objects is disclosed in Pavithra et al. based on highly related categories would read on the basing the determination to include textual content or not in a description.); and
basing the determination not to include in the textual content any description based on the second group of two or more events of the second physical object on the second mathematical object (see ¶ 1 of III. Proposed Work and ¶ 1-4 of III. Proposed Work: A. Text Analysis citations as in claim 11, above and further Figure 2 (Similarity Graph Construction) and ¶ 1 of III. Proposed work: C. Microblog Clique Generations citations as in limitation above. More specifically: “… If the similarity value higher than or equal to threshold value, then two posts are highly related, else not. One post is compared with all other posts in the collection. Based on the computed similarity matrix, similarity graph is constructed…”
Here, the Examiner notes that the determination to include or not in textual content description based on events associated with mathematical objects is disclosed in Pavithra et al. based on highly related categories would read on the basing the determination to include textual content or not in a description.).
Zhang et al. and Pavithra et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in event/object analysis. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. to incorporate the teachings of Pavithra et al. wherein the operations further comprise: analyzing the mathematical objects corresponding to the first group of one or more events to determine a first mathematical object in the mathematical space, the first mathematical object differs from any mathematical object of the mathematical objects corresponding to the first group of one or more events; analyzing the mathematical objects corresponding to the second group of one or more events to determine a second mathematical object in the mathematical space, the second mathematical object differs from any mathematical object of the mathematical objects corresponding to the second group of one or more events; basing the determination to include in the textual content the description based on the first group of one or more events of the first object on the first mathematical object; and basing the determination not to include in the textual content any description based on the second group of one or more events of the second object on the second mathematical object which provides the benefit of improving performance of clustering algorithms helpful in the monitoring of events by public organizations (Abstract of Pavithra et al.).
Claim 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 20190366153 A1) as applied to claim 1 above, and further in view of Ray (US 11947898 B2).
Regarding claim 13, Zhang et al. teaches the limitations as in claim 11, above.
However, Zhang et al. does not explicitly teach, but Pavithra et al. does teach:
13. The non-transitory computer readable medium of claim 1,
wherein
the textual content is associated with a writer persona (see ¶ starting at Col. 3, line 17: “(19) A method and apparatus of a device that generates one or more content briefs is described. In one embodiment, the device collects content based on an inputted strategy. In one embodiment, the strategy can be one or more of a topic, context, persona, channel, and/or a combination thereof. With the inputted strategy, the device collects content by crawling the internet for available content according to the strategy. In addition, the device stores the content.”),
the determination to include in the textual content the description based on the first group of two or more events of the first physical object is based on the writer persona (see ¶ starting at Col. 4, line 46: “(26) The content generation server 104, in one embodiment, generates the content briefs and/or headlines using a strategy that can be tailored for a particular company or organization. In one embodiment, the content generation device 108 generates the content briefs and/or headlines based on a strategy that is created by the particular company or organization that wishes to have the content briefs and/or headlines generated. In one embodiment, the strategy can include particular personas, topics, contexts, and/or channels that can be used to guide the collection of content for the content brief and/or headline generation. In this embodiment, one or more persona can include a person, personality, identity, organization, or another type of entity that are to be included in the content collection. For example, and in one embodiment, for a media company, persona can be different media people that review media, such as online bloggers, social media influencers, print media reviewers, etc. In a further embodiment, a topic can be a set of one of more subjects that are to be included in the content collection and/or to guide the content collection.”
Here, the Examiner notes that the generation of content briefs using a strategy including particular personas read on the determination to include or not in the textual content description based on events associated with writer persona.), and
the determination not to include in the textual content any description based on the second group of two or more events of the second physical object is based on the writer persona (see ¶ starting at Col. 3, line 17 and ¶ starting at Col. 4, line 46 citations as in limitations above.
Here, the Examiner notes that the generation of content briefs using a strategy including particular personas read on the determination to include or not in the textual content description based on events associated with writer persona.).
Zhang et al. and Ray are considered to be analogous to the claimed invention because they are in the same field of endeavor in content generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. to incorporate the teachings of Ray of wherein the textual content is associated with a writer persona, the determination to include in the textual content the description based on the first group of one or more events of the first object is based on the writer persona, and the determination not to include in the textual content any description based on the second group of one or more events of the second object is based on the writer persona. which provides the benefit of allowing the public relations department to generate a greater number of content briefs for different content creators (Col. 6, lines 36-40 of Ray).
Claim 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 20190366153 A1) as applied to claim 1 above, and further in view of López Felip et al. (US 20220038643A1).
Regarding claim 14, Jolly et al. teaches the limitations as in claim 1, above.
However, Zhang et al. does not explicitly teach, but López Felip et al. does teach:
14. The non-transitory computer readable medium of claim 1,
wherein
the textual content is associated with a prospective audience (see ¶ [0022]: “Aspects and embodiments of the present disclosure here allow, based on the heterogeneous algorithms set of the model graph, one or more of the following, for a series of advantageous functionalities in respect to the prior-art. First, automatic detection of patterns of play in the sporting event; that are of direct interest for the coach and others—in lieu of traditional statistical trends or numeric values that require interpretation. Second, it allows for automatic classification of the patterns based on data revealed in the process of the detection (e.g., the players involved, location on the field of play, type) and/or attributes of the type of event (e.g., formation, offensive or defensive tactical fundamentals, etc.). And thirdly, it allows for automatic graphic representation for each pattern detected and generated in an augmented video which communicates the nature of the pattern. Such video has value for purposes such as performance enhancement, review, and improvement; tendency analysis for understanding opponents; education and training of tactical principles; and content-generation for media and direct fan consumption. Further, it may do so in a way constrained by computational processing capacity rather than human attentional capacity, and with objective analysis. These innovations allow for sporting events to be analyzed more quickly, in greater width and depth, and in parallel, compared to previous methods, saving time and resources of analysts while increasing the level of engagement with the content. Therefore, the present aspects and embodiments improve the efficiency, efficacy, and cost-effectiveness of content generation, communication of gameplay event characteristics, and performance review and improvement for any type of sports organization.”),
the determination to include in the textual content the description based on the first group of one or more events of the first physical object is based on the prospective audience (audience (see ¶ [0022]: “Aspects and embodiments of the present disclosure here allow, based on the heterogeneous algorithms set of the model graph, one or more of the following, for a series of advantageous functionalities in respect to the prior-art. First, automatic detection of patterns of play in the sporting event; that are of direct interest for the coach and others—in lieu of traditional statistical trends or numeric values that require interpretation…”), and
the determination not to include in the textual content any description based on the second group of one or more events of the second physical object is based on the prospective audience (see ¶ [0022]: “Aspects and embodiments of the present disclosure here allow, based on the heterogeneous algorithms set of the model graph, one or more of the following, for a series of advantageous functionalities in respect to the prior-art. First, automatic detection of patterns of play in the sporting event; that are of direct interest for the coach and others—in lieu of traditional statistical trends or numeric values that require interpretation…”).
Zhang et al. and López Felip et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in content generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. to incorporate the teachings of López Felip et al. of wherein the textual content is associated with a prospective audience, the determination to include in the textual content the description based on the first group of one or more events of the first object is based on the prospective audience and the determination not to include in the textual content any description based on the second group of one or more events of the second object is based on the prospective audience which provides the benefit of improving the efficiency, efficacy, and cost-effectiveness of content generation([0022] of López Felip et al.).
Claim 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang et al. (US 20190366153 A1) as applied to claim 1 above, and further in view of Chan et al. (US 20210089779 A1).
Regarding claim 21, Zhang et al. teaches the limitations as in claim 16, above.
Zhang et al. further teaches:
21. (New) The non-transitory computer readable medium of claim 1, wherein the image data is a video of a basketball game (see ¶ [0009 and 0070] citations as in claim 1, above. More specifically: “…receiving an input video of a ball game and a location of a shooter…” and further ¶ [0008]: “Some embodiments of the present invention include methods and systems for mobile device-based real-time detection, analysis and recording of basketball shot attempts. The method includes, but is not limited to, the steps of tracking ball(s) and shooter(s) in an input video…”),
the first physical object is a first basketball player (see Fig. 9B (players: 904-905, ball: 902, and basket/hoop: 980), ¶ [0085 and 0154] citations as in claim 1 above. More specifically: “[0085] …Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures... [0154] …Trajectory 903 represents a pass from player 905 to player 904, where the ball does not move above any of these two player's upper bodies, or come close to basket 980…”
Here, the first physical object is analogous to player 904, while the second physical object is analogous to player 905.),
the second physical object is a second basketball player (see Fig. 9B (players: 904-905, ball: 902, and basket/hoop: 980), ¶ [0085 and 0154] citations as in claim 1 above. More specifically: “[0085] …Feature extraction in turn enables the segmentations or identifications of image areas representing balls and players, and further analysis to determine player body postures... [0154] …Trajectory 903 represents a pass from player 905 to player 904, where the ball does not move above any of these two player's upper bodies, or come close to basket 980…”
Here, the first physical object is analogous to player 904, while the second physical object is analogous to player 905.),
the first group of two or more events includes shots of a basketball to a hoop by the first basketball player (see ¶ [0020 and 0099-0100]: “[0020] In some embodiments, the shot event is selected from the group consisting of dribble event, jump event, catch-ball event, ball-leave-hand event, one-two leg jump, shooter's foot-on-ground movement, and the shot type is selected from the group consisting of layup, regular shot, dribble-pull-up, off-the-move, and catch-and-shoot. [0073] FIG. 1B is a flow diagram 190 providing a process overview of using a mobile device-based NEX system 150 to generate shot analytics and statistics, according to one embodiment of the present invention. This exemplary process takes as inputs a video segment or video stream, and/or a shooter's location in any given frame of the video input. Through new and novel methods for computer vision and algorithmic analysis, systems and devices implemented according to embodiments of the present invention extract various shot analytics, including, but are not limited to, shot type, release time, release angle, shooter body bend angle, leg bend ratio, moving speed and direction, and height of a jump event. The input video may be a live-stream, or an off-line recording, and may be a single perspective video, also known as a monocular video. [0099] With filtered flow and shot information 315, the NEX system may apply the remaining process steps in FIG. 3 to determine one or more shot events occurring before the ball-from-shooter time, and to generate one or more shot analytics 185 based on the one or more shot events, the shooter posture flow, and the related ball flow. In this disclosure, a “shot event” refers to player actions leading up to a shot attempt. That is, a shot event describes player movements before the ball leaves the shooter's hand in a shot attempt. A shot event may occur right before a shot is launched, or some time shortly before the shot is launched. [0100] In process step 320 shown in FIG. 3, several exemplary shot events are detected, for example, a dribble event, a jump event, a catch-ball event, as well as shooter movement in image space. Detected shot events, shooter movement in image space, shooter posture flow, and ball-shoot-from-hand time are used as input 325 to further processing steps 329, 330 and optionally 331, to determine one or more shot analytics.”
Here, the Examiner notes that the indication of a first group of two or more events is read by disclosures in Zhang et al. regarding events associated with player 904 and the ball (e.g., shot event: dibble event, jump event, catch-ball event, etc.).,
the second group of two or more events includes shots of the basketball to the hoop by the second basketball player (see ¶ [0020 and 0099-0100] citations as in limitation above and further ¶ [0150]: “In table 810, raw information is divided into ball information 812, shooter information 814, events information 816, and scene information 818. For balls extracted from the input video, one or more ball flow and trajectories may be identified, and shot attempts may be determined based on the ball trajectories and their positions relative to the hoop. For the shooter, pose information may be determined from, for example, 18 key points on the body. Following a shot attempt trajectory, shooter poses may be detected in the region around the ball, and tracked as shooter poses. In some embodiments, more than one player may be present, and shooter information 814 may refer to player pose information and player posture flow as discussed with reference to FIGS. 1B to 4. In addition, shooter information 814 may be correlated with ball information 812 to determine different shot events such as ball-leave-hand, jump, dribble, and catch-ball events. Scene information 818 includes how hoop, court, and other relevant objects of interests are placed within the image domain, including hoop detection information and how court is placed in the image. Such scene information may be combined with other ball, shooter, and events information to generate shot analytics and/or game analytics, such as determining whether a shot is a 3-pointer or not.”
Here, the Examiner notes that the indication of a second group of two or more events is read by disclosures in Zhang et al. regarding events associated with the second player 905 and the ball OR by disclosures of other scene information (e.g., hoop, court or other relevant objects present in the image).), and
However, Zhang et al. does not explicitly teach, but Chan et al. does teach:
the operations further comprise:
determining, based on an analysis of the video, a number of points scored by the first basketball player and a number of points scored by the second basketball player (see ¶ [0207-0208]: “[0207] FIG. 41 relates to an offering referred to as “inSight.” This offering allows pushing of relevant stats to fans' mobile devices 4104. For example, if player X just made a three-point shot from the wing, this would show statistics about how often he made those types of shots 4108, versus other types of shots, and what types of play actions he typically made these shots off of inSight does for hardcore fans what Eagle (the system described above) does for team analysts and coaches. Information, insights, and intelligence may be delivered to fans' mobile devices while they are seated in the arena. This data is not only beautiful and entertaining, but is also tuned into the action on the court. For example, after a seemingly improbable corner three by a power forward, the fan is immediately pushed information that shows the shot's frequency, difficulty, and likelihood of being made. In embodiments, the platform features described above as “Eagle,” or a subset thereof may be provided, such as in a mobile phone form factor for the fan. An embodiment may include a storyboard stripped down, such as from a format for an 82″ touch screen to a small 4″ screen. Content may be pushed to a device that corresponds to the real time events happening in the game. Fans may be provided access to various effects (e.g., DataFX features described herein) and to the other features of the methods and systems disclosed herein. [0208] FIGS. 42 and 43 show touchscreen product interface elements 4202, 4204, 4208, 4302 and 4304. These are essentially many different skins and designs on the same basic functionality described throughout this disclosure. Advanced stats are shown in an intuitive large-format touch screen interface. A touchscreen may act as a storyboard for showing various visualizations, metric and effects that conform to an understanding of a game or element thereof. Embodiments include a large format touch screen for commentators to use during a broadcast. While InSight serves up content to a fan, the Storyboard enables commentators on TV to access content in a way that helps them tell the most compelling story to audiences.”);
PNG
media_image1.png
550
800
media_image1.png
Greyscale
basing the determination to include in the textual content the description based on the first group of two or more events of the first physical object on the number of points scored by the first basketball player (see ¶ [0207]: “[0207] FIG. 41 relates to an offering referred to as “inSight.” This offering allows pushing of relevant stats to fans' mobile devices 4104. For example, if player X just made a three-point shot from the wing, this would show statistics about how often he made those types of shots 4108, versus other types of shots, and what types of play actions he typically made these shots off of inSight does for hardcore fans what Eagle (the system described above) does for team analysts and coaches. Information, insights, and intelligence may be delivered to fans' mobile devices while they are seated in the arena. This data is not only beautiful and entertaining, but is also tuned into the action on the court. For example, after a seemingly improbable corner three by a power forward, the fan is immediately pushed information that shows the shot's frequency, difficulty, and likelihood of being made. In embodiments, the platform features described above as “Eagle,” or a subset thereof may be provided, such as in a mobile phone form factor for the fan. An embodiment may include a storyboard stripped down, such as from a format for an 82″ touch screen to a small 4″ screen. Content may be pushed to a device that corresponds to the real time events happening in the game. Fans may be provided access to various effects (e.g., DataFX features described herein) and to the other features of the methods and systems disclosed herein.”); and
basing the determination not to include in the textual content any description based on the second group of two or more events of the second physical object on the number of points scored by the second basketball player (see ¶ [0207] citation as in limitation above. Here, the Examiner notes that in Chan et al., the shot analysis information (i.e., including the points by the player) is displayed for the player that just made a point shot (not the other players).).
PNG
media_image2.png
576
812
media_image2.png
Greyscale
Zhang et al. and Chan et al. are considered to be analogous to the claimed invention because they are in the same field of endeavor in content generation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified Zhang et al. to incorporate the teachings of Chan et al. of the operations further comprise: determining, based on an analysis of the video, a number of points scored by the first basketball player and a number of points scored by the second basketball player; basing the determination to include in the textual content the description based on the first group of two or more events of the first physical object on the number of points scored by the first basketball player; and basing the determination not to include in the textual content any description based on the second group of two or more events of the second physical object on the number of points scored by the second basketball player which provides the benefit of enabling the access of content in a way that helps tell the most compelling story to audiences([0207] of Chan et al.).
Conclusion
Examiner Notes
The Examiner cites particular columns and line numbers in the references as applied to the claims above for the convenience of the Applicant. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested that, in preparing responses, the Applicant fully considers the references in its entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or as disclosed by the Examiner.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
This action is a final rejection and closes the prosecution of this application. Applicant’s reply under 37 CFR 1.113 to this action is limited to an appeal to the Patent Trial and Appeal Board, an amendment complying with the requirements set forth below, or a request for continued examination (RCE) to reopen prosecution where permitted. Please note that the Office also offers initiatives that are available to applicants after the close of prosecution. See https://www.uspto.gov/patents/initiatives/uspto-patent-applications-iniatives-timeline for more information.
General information on the Patent Trial and Appeal Board is available at: www.uspto.gov/patents/ptab. The information at this page includes guidance on time limited options that may assist the applicant contemplating appealing an examiner’s rejection. It also includes information on pro bono (free) legal services and advice available for those who are under-resourced and considering an appeal at: https://www.uspto.gov/patents/ptab/free-legal-assistance. The page is best reviewed promptly after applicant has received a final rejection or the claims have been twice rejected because some of the noted assistance must be requested within one month from the date of the latest rejection. See MPEP § 1204 for more information on filing a notice of appeal.
If applicant should desire to appeal any rejection made by the examiner, a Notice of Appeal must be filed within the period for reply. The Notice of Appeal must be accompanied by the fee required by 37 CFR 41.20(b)(1). The current fee amount is available at: www.uspto.gov/Fees.
If applicant should desire to file an after-final amendment, entry of the proposed amendment cannot be made as a matter of right unless it merely cancels claims or complies with a formal requirement made in a previous Office action. Amendments touching the merits of the application which otherwise might not be proper may be admitted upon a showing of good and sufficient reasons why they are necessary and why they were not presented earlier.
A reply under 37 CFR 1.113 to a final rejection must include cancellation of or appeal from the rejection of, each rejected claim. The filing of an amendment after final rejection, whether or not it is entered, does not stop the running of the statutory period for reply to the final rejection unless the examiner holds all of the claims to be in condition for allowance.
If applicant should desire to continue prosecution in a utility or plant application filed on or after May 29, 2000 and have the finality of this Office action withdrawn, an RCE under 37 CFR 1.114 may be filed within the period for reply. See MPEP § 706.07(h) for more information on the requirements for filing an RCE.
The application will become abandoned unless a Notice of Appeal, an after final reply that places the application in condition for allowance, or an RCE has been filed properly within the period for reply, or any extension of this period obtained under either 37 CFR 1.136(a) or (b).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Keisha Y Castillo-Torres whose telephone number is (571)272-3975. The examiner can normally be reached Monday - Friday, 9:00 am - 4:00 pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre-Louis Desir can be reached on (571)272-7799. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
Keisha Y. Castillo-Torres
Examiner
Art Unit 2659
/Keisha Y. Castillo-Torres/Examiner, Art Unit 2659
/PIERRE LOUIS DESIR/Supervisory Patent Examiner, Art Unit 2659
Prosecution Insights