DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending in the application.
Claim Objections
The following claims are objected to because of the following reasons.
Claim 2 3rd line the first “automatically detecting” should be “detecting”.
Claim 4 1st line “analyzing” should be “detecting”.
Claim 5 1st-2nd line “analyzing in the video frames of the first sequence, the video frame region” is suggested to recite “analyzing the video frames of the first sequence to detect the video frame region”.
Claim 5 8th line “wherein the video frame region” should be “wherein detecting the video frame region”.
Claim 6 2nd line “wherein analyzing in the video frames” should be ““wherein analyzing the video frames”.
Claim 7 4th-6th line the two “wherein” clauses are suggested to recite “wherein analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard is based on detecting a set of edges, wherein detecting the video frame region comprises:”.
Claim 8 1st-3rd line is suggested to recite “The computer-implemented method of claim 1, wherein analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard is based on detecting a set of edges, wherein detecting the video frame region comprises:”
Claim 9 2nd line “first sequence, the video frame region” should be “first sequence to detect the video frame region”.
Claim 9 4th line “the computing-implement method further comprising” should be “the computer-implemented method further comprises”.
Claim 10 5th line “receiving a video streams” should be “receiving video streams”.
Claim 12 first 2 lines the limitation “wherein receiving the first sequence comprises receiving a video streams that include the first sequence” has already included in claim 10.
Claim 13 1st-2nd line “wherein analyzing the video frame region that depicts the video frame region that depicts the scoreboard comprises” should be “wherein detecting the video frame region that depicts the scoreboard comprises”.
Claim 14 2nd line “wherein the video frame region” should be “wherein detecting the video frame region”.
Claim 14 (page 7 3rd line) “wherein the video frame region” should be “wherein detecting the video frame region”.
Claim 16 (page 7 last 2 lines) should be “sequence to detect the video frame region that depicts the scoreboard is based on detecting a set of edges, wherein detecting the video frame region comprises:”.
Claim 17 1st – 4th line is suggested to recite “The non-transitory computer readable medium of claim 10, wherein analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard is based on detecting a set of edges, wherein detecting the video frame region comprises”.
Claim 18 1st – 3rd lines is suggested to recite “The non-transitory computer readable medium of claim 10, wherein detecting the video frame region that depicts the scoreboard is based on detecting a set of edges,”.
Claim 18 the 4th line “wherein the computing readable medium further includes” is suggested to recite “wherein the operations further include”.
Claim 20 5th line “a video streams” should be “video streams”.
Claim 20 7th line “the first sequence, the video frame region” should be “the first sequence to detect the video frame region”.
Claim 20 10th line “to detect” should be removed.
Claim 20 12th line “at least a predetermined number” should be “at least a number”.
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: Claim 20: “a processing unit” and associated function. The corresponding structure disclosed in the specification is FIG. 10 #1004, para. [0234] (PGPub) general purpose processor(s) (e.g., microprocessors) and/or specialized processor(s) (e.g., application specific integrated circuits). The corresponding algorithm associated with the function is disclosed in FIG. 2-9, para. [0074]-[0229] (PGPub).
To claim a means for performing a specific computer-implemented function and then to disclose only a general purpose computer as the structure designed to perform that function amounts to pure functional claiming. Aristocrat, 521 F.3d 1328 at 1333, 86 USPQ2d at 1239. In this instance, the structure corresponding to a 35 U.S.C. 112(f) claim limitation for a computer-implemented function must include the algorithm needed to transform the general purpose computer or microprocessor disclosed in the specification. Aristocrat, 521 F.3d at 1333, 86 USPQ2d at 1239; Finisar Corp. v. DirecTV Group, Inc., 523 F.3d 1323, 1340, 86 USPQ2d 1609, 1623 (Fed. Cir. 2008); WMS Gaming, Inc. v. Int’l Game Tech., 184 F.3d 1339, 1349, 51 USPQ2d 1385, 1391 (Fed. Cir. 1999); Rain Computing, Inc. v. Samsung Electronics America Co., 989 F.3d 1002, 1007-8, 2021 USPQ2d 284 (Fed. Cir. 2021). (MPEP II (b)).
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claim 1 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 1 of U.S. Patent No. US 12,283,055 B2.
Listed in the following table is a limitation-to-limitation comparison of the examined claim 1 and the conflicting claim 1.
Application being examined 19/082,874 (hereafter ‘874 application)
Conflicting patent 12,283,055 B2 (hereafter ‘055 patent)
1. A computer-implemented method to detect a scoreboard within a video frame region, the computer-implemented method comprising:
receiving video streams including a first sequence of video frames and one or more additional video frames;
analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard;
executing a machine-driven pattern recognition procedure to detect, in one or more of the additional video frames, the video frame region that depicts the scoreboard;
based on results of the machine-driven pattern recognition procedure, determining whether at least a number of video frames, from among the video streams, each depict the scoreboard within the video frame region; and
responsive to the determination being that at least the number of video frames exceeds a predetermined number, from among the video streams, each depict the scoreboard within the video frame region, (i) determining that the detected video frame region depicts the scoreboard was accurate and (ii) responsive to the determining that the detected video frame region depicts the scoreboard was accurate, extracting a sport score from the detected scoreboard region from the video frame region of a video frame of the video streams.
1. A method for automatically detecting a video frame region that depicts a scoreboard, the method comprising:
receiving, by a computing system, a plurality of video frames including a first sequence of video frames and further including additional video frames, wherein the computing system comprises a processing unit;
engaging by the computing system, in a detection process to detect, in the video frames of the first sequence, the video frame region that depicts the scoreboard;
executing, by the computing system, a machine-driven pattern recognition procedure to detect, in one or more of the additional video frames, the video frame region that depicts the scoreboard;
based on results of the machine-driven pattern recognition procedure, making a determination, by the computing system, of whether at least a predetermined number of video frames, from among the plurality of video frames, each depict the scoreboard within the video frame region; and
responsive to the determination being that at least the predetermined number of video frames, from among the plurality of video frames, each depict the scoreboard within the video frame region, the computing system (i) determining that the detecting that the video frame region depicts the scoreboard was accurate and (ii) responsive to the determining that the detecting that the video frame region depicts the scoreboard was accurate, extracting a sport score from the detected scoreboard region from the video frame region of a video frame of the plurality of video frames.
Claim 1 of ‘055 patent teaches every limitation recited in claim 1 of ‘874 application.
Claim 10 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 10 of U.S. Patent No. US 12,283,055 B2.
Listed in the following table is a limitation-to-limitation comparison of the examined claim 10 and the conflicting claim 10.
Application being examined 19/082,874 (hereafter ‘874 application)
Conflicting patent 12,283,055 B2 (hereafter ‘055 patent)
10. A non-transitory computer readable medium having stored thereon instructions executable by a processing unit to cause a computing system to perform operations for automatically detecting, in a sequence of video frames, a video frame region that depicts a scoreboard, the operations comprising:
receiving a video streams including a first sequence of video frames and further including additional video frames;
analyzing in the video frames of the first sequence to detect the video frame region that depicts the scoreboard;
executing a machine-driven pattern recognition procedure to detect, in one or more of the additional video frames, the video frame region that depicts the scoreboard;
based on results of the machine-driven pattern recognition procedure, determining whether a number of video frames, from among the video streams, each depict the scoreboard within the video frame region; and
responsive to the determination being that the number of video frames exceeds a predetermined number, from among the video streams, each depict the scoreboard within the video frame region, (i) determining that the detecting that the video frame region depicts the scoreboard was accurate and (ii) responsive to the determining that the detecting that the video frame region depicts the scoreboard was accurate, extracting a first sport score from the detected scoreboard region from the video frame region of a video frame of the video streams.
10. A non-transitory computer readable medium having stored thereon instructions executable by a processing unit to cause a computing system to perform operations for automatically detecting, in a sequence of video frames, a video frame region that depicts a scoreboard, the operations comprising:
receiving a plurality of video frames including a first sequence of video frames and further including additional video frames;
engaging in a detection process to detect, in the video frames of the first sequence, the video frame region that depicts the scoreboard;
executing a machine-driven pattern recognition procedure to detect, in one or more of the additional video frames, the video frame region that depicts the scoreboard;
based on results of the machine-driven pattern recognition procedure, making a determination of whether at least a predetermined number of video frames, from among the plurality of video frames, each depict the scoreboard within the video frame region; and
responsive to the determination being that at least the predetermined number of video frames, from among the plurality of video frames, each depict the scoreboard within the video frame region, (i) determining that the detecting that the video frame region depicts the scoreboard was accurate and (ii) responsive to the determining that the detecting that the video frame region depicts the scoreboard was accurate, extracting a first sport score from the detected scoreboard region from the video frame region of a video frame of the plurality of video frames.
Claim 10 of ‘055 patent teaches every limitation recited in claim 10 of ‘874 application.
Claim 20 is rejected on the ground of nonstatutory double patenting as being unpatentable over claim 20 of U.S. Patent No. US 12,283,055 B2.
Listed in the following table is a limitation-to-limitation comparison of the examined claim 20 and the conflicting claim 20.
Application being examined 19/082,874 (hereafter ‘874 application)
Conflicting patent 12,283,055 B2 (hereafter ‘055 patent)
20. A computing system comprising: a processing unit coupled to a non-transitory data storage storing program instructions that when executed by the processing unit implement operations to detect a video frame region that depicts a scoreboard, the operations comprising:
receiving a video streams including a first sequence of video frames and further including additional video frames,
analyzing the video frames of the first sequence, the video frame region that depicts the scoreboard,
executing a machine-driven pattern recognition procedure to detect, in one or more of the additional video frames to detect the video frame region that depicts the scoreboard,
based on results of the machine-driven pattern recognition procedure, determining whether at least a predetermined number of video frames, from among the video streams, each depict the scoreboard within the video frame region, and
responsive to the determination being that the number of video frames exceeds a predetermined number, from among the video streams, each depict the scoreboard within the video frame region, (i) determining that the detected video frame region depicts the scoreboard was accurate and (ii) responsive to the determining that the detected video frame region depicts the scoreboard was accurate, extracting a first sport score from the detected scoreboard region from the video frame region of a video frame of the video streams.
20. A computing system comprising: a processing unit; non-transitory data storage; and program instructions stored in the non-transitory data storage and executable by the processing unit to carry out operations for automatically detecting a video frame region that depicts a scoreboard, the operations comprising:
receiving a plurality of video frames including a first sequence of video frames and further including additional video frames,
engaging in a detection process to detect, in the video frames of the first sequence, the video frame region that depicts the scoreboard,
executing a machine-driven pattern recognition procedure to detect, in one or more of the additional video frames, the video frame region that depicts the scoreboard,
based on results of the machine-driven pattern recognition procedure, making a determination of whether at least a predetermined number of video frames, from among the plurality of video frames, each depict the scoreboard within the video frame region, and
responsive to the determination being that at least the predetermined number of video frames, from among the plurality of video frames, each depict the scoreboard within the video frame region, (i) determining that the detecting that the video frame region depicts the scoreboard was accurate and (ii) responsive to the determining that the detecting that the video frame region depicts the scoreboard was accurate, extracting a first sport score from the detected scoreboard region from the video frame region of a video frame of the plurality of video frames.
Claim 20 of ‘055 patent teaches every limitation recited in claim 20 of ‘874 application.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-5, 7-14 and 16-19 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Analysis for claim 1 is provided in the following. Claim 1 is reproduced in the following (annotation added):
1. A computer-implemented method to detect a scoreboard within a video frame region, the computer-implemented method comprising:
(a) receiving video streams including a first sequence of video frames and one or more additional video frames;
(b) analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard;
(c) executing a machine-driven pattern recognition procedure to detect, in one or more of the additional video frames, the video frame region that depicts the scoreboard;
(d) based on results of the machine-driven pattern recognition procedure, determining whether at least a number of video frames, from among the video streams, each depict the scoreboard within the video frame region; and
(e) responsive to the determination being that at least the number of video frames exceeds a predetermined number, from among the video streams, each depict the scoreboard within the video frame region, (i) determining that the detected video frame region depicts the scoreboard was accurate and (ii) responsive to the determining that the detected video frame region depicts the scoreboard was accurate, extracting a sport score from the detected scoreboard region from the video frame region of a video frame of the video streams.
Step 1: Evaluating whether the claim belongs to one of the statutory categories.
Claim 1 recites a plurality of acts. Thus, the claim is directed to a process, which is one of the statutory categories of invention (Step 1: YES).
Step 2A Prong One: Evaluating whether the claim recites a judicial exception (an abstract idea enumerated in the 2019 PEG, a law of nature, or a natural phenomenon). If no exception is recited, the claim is eligible. This concludes the eligibility analysis. If the claim recites an exception, go to Step 2A Prong Two.
Claim 1 recites an abstract idea in the mental processes group. In claim 1, under broadest reasonable interpretation, the function in steps (b)-(e) can be practically performed in the human mind. (Step 2A Prong One YES)
Step 2A Prong Two: Evaluating whether the claim recites additional elements that integrate the exception into a practical application of the exception. This evaluation is performed by (a) identifying whether there are any additional elements recited in the claim beyond the judicial exception, and (b) evaluating those additional elements individually and in combination to determine whether the claim as a whole integrates the exception into a practical application. If the answer to (a) is YES and (b) is NO, go to Step 2B; if the answer to (a) and (b) is YES, go to PATHWAY B, i.e., the claim is not directed to a judicial exception and the claim is eligible.
In claim 1, step a) “receiving video streams” recites data acquisition and collection, an insignificant extra-solution activity. Step (c) further recites “executing a machine-driven pattern recognition procedure to detect”. Here the “machine-driven pattern recognition procedure” is recited in a high level of generality. The recited “machine-driven pattern recognition procedure” merely adds a generic computer component to perform the act and therefore fails to provide an improvement to the technology or technical field. See MPEP 2106.05(a). Therefore, these additional elements do not integrate the abstract idea into a practical application. (Step 2A Prong Two NO).
Step 2B: This part of the eligibility analysis evaluates whether the claim as a whole amounts to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
In claim 1, the recited “machine-driven pattern recognition procedure” is at best equivalent of merely adding the words “apply it” to the judicial exception. Step a) recites data acquisition and collection, an insignificant extra-solution activity, elements that should be re-evaluated in Step 2B. Step a) recites receiving data in a high level of generality, which is well-understood, routine and conventional activity. These additional elements, taken individually and in combination, do not result in the claim, as a whole, amounting to significantly more than the abstract idea. See MPEP 2106.05. (Step 2B: NO). Claim 1 is not eligible.
Claim 2 recites: The computer-implemented method of claim 1, wherein receiving the video streams comprises receiving a real-time broadcast feed including the video streams, and wherein automatically detecting the video frame region comprises automatically detecting the video frame region while receiving the real-time broadcast feed.
Under broadest reasonable interpretation, the “detecting” act can be practically performed in human mind and therefore recites mental processes. Claim 2 is not eligible.
Claim 3 recites: The computer-implemented method of claim 1, wherein the video streams depict a sport- related event, the computer-implemented method further comprising:
determining the sport-related event depicted by the video streams;
based on the determined sport-related event, determining a quantity of video frames to use as basis for automatically detecting the video frame region; and
selecting from among the video streams, the first sequence based on the selected sequence including the determined quantity of video frames.
The recited “determining the sport-related event” act, the “determining a quantity” act and the “selecting” act, can be practically performed in human mind, under broadest reasonable interpretation. Therefore claim 3 recites mental processes. Claim 3 is not eligible.
Claim 4 recites: The computer-implemented method of claim 1, wherein analyzing the video frame region that depicts the scoreboard comprises engaging in canny edge detection to detect a set of edges, wherein detecting the video frame region is based on the detected set of edges. Performing edge detection using Canny edge detection recites mathematical concepts. Claim 4 is not eligible.
Claim 5 recites (annotation added): The computer-implemented method of claim 1, wherein analyzing in the video frames of the first sequence, the video frame region that depicts the scoreboard is based on detecting a set of edges,
wherein the first sequence includes a first video frame and a second video frame,
wherein the set of edges comprises first edges and second edges,
wherein detecting the set of edges comprises detecting the first edges in the first video frame and detecting the second edges in the second video frame, and
wherein the video frame region that depicts the scoreboard comprises (i) applying a logical conjunction on the detected first and second edges to produce an output that indicates common edges detected in both the first video frame and the second video frame, and (ii) deeming the common edges to be the edges of a subset of edges, wherein detecting the video frame region is based on the subset of edges.
The recited acts of “detecting the first edges”, “detecting the second edges”, and “deeming the common edges to be the edges of a subset of edges, wherein detecting the video frame region is based on the subset of edges”, under broadest reasonable interpretation, can be practically performed in human mind and therefore recite mental processes. The “applying a logical conjunction on the detected first and second edges to produce an output” act recites mathematical concepts. Claim 5 is not eligible.
Claim 7 recites (annotation added): The computer-implemented method of claim 1, wherein the video frames of the first sequence depict image elements including (i) scoreboard image elements that are unchanging across the video frames of the first sequence and (ii) other image elements that change across the video frames of the first sequence, wherein analyzing the video frames of the first sequence, the video frame region that depicts the scoreboard is based on detecting a set of edges, wherein analyzing the video frame region comprises:
based on the edges of the set, engaging in a filtering process to determine actual edges of the scoreboard image elements, wherein the filtering process comprises (i) recognizing edges, from among the edges of the set, based on the recognized edges having a predetermined characteristic and (ii) deeming the recognized edges to be the actual edges of the scoreboard image elements; and
detecting, based on the determined actual edges of the scoreboard image elements, the video frame region that depicts the scoreboard.
The recited acts of “detecting a set of edges”, “recognizing edges”, “deeming the recognized edges to be the actual edges of the scoreboard image elements” and “detecting, based on the determined actual edges of the scoreboard image elements, the video frame region that depicts the scoreboard”, under broadest reasonable interpretation, can be practically performed in human mind and therefore recite mental processes. Claim 7 is not eligible.
Claim 8 recites (annotation added): The computer-implemented method of claim 1, wherein analyzing the video frames of the first sequence, the video frame region that depicts the scoreboard is based on detecting a set of edges, wherein analyzing the video frame region comprises:
engaging in a morphological-dilatation process to identify a substantially uniform region based on the edges of the set; and based on the substantially uniform region, determining the video frame region that depicts the scoreboard.
The acts of “detecting a set of edges” and “determining the video frame region that depicts the scoreboard”, under broadest reasonable interpretation, can be practically performed in human mind and therefore recite mental processes. The act of “engaging in a morphological-dilatation process to identify a substantially uniform region based on the edges of the set” recites mathematical concepts. Claim 8 is not eligible.
Claim 9 recites (annotation added): The computer-implemented method of claim 1, wherein analyzing the video frames of the first sequence, the video frame region that depicts the scoreboard is based on detecting a set of edges,
wherein the computing-implemented method further comprising accessing region-information indicating a predefined frame region that is not a candidate for depicting the scoreboard,
wherein analyzing the video frames comprises, based on the region-information, excluding, from the analyzing of the video frames, detection of edges in the predefined frame region of the video frames of the first sequence, and
wherein detecting the set of edges comprises detecting the set of edges within one or more frame regions other than the predefined frame region.
The above recited acts of “detecting a set of edges”, “accessing region-information”, “excluding … detection of edges” and “detecting the set of edges within one or more frame regions other than the predefined frame region”, under broadest reasonable interpretation, can be practically performed in human mind and therefore recite mental processes. Claim 9 is not eligible.
Claim 10, an independent medium claim, recites similar steps as in claim 1 and therefore recites abstract idea without significantly more. Claim 10 further recites computer system elements, such as computer readable medium and computing system. These elements are recited in a high level of generality and merely add generic computer components to perform the acts and therefore fail to provide an improvement to the technology or technical field. Claim 10 is not eligible.
Per Claim 11, similar analysis is applied to claim 2. Claim 11 is not eligible.
Per Claim 12, similar analysis is applied to claim 3. Claim 12 is not eligible.
Per Claim 13, similar analysis is applied to claim 4. Claim 13 is not eligible.
Per Claim 14, similar analysis is applied to claim 5. Claim 14 is not eligible.
Per Claim 16, similar analysis is applied to claim 7. Claim 16 is not eligible.
Per Claim 17, similar analysis is applied to claim 8. Claim 17 is not eligible.
Per Claim 18, similar analysis is applied to claim 9. Claim 18 is not eligible.
Claim 19 recites: The non-transitory computer readable medium of claim 10, further comprising: carrying out, by the computing system, an action based on the detecting of the video frame region that depicts the scoreboard.
The recited act “carrying out … an action” can be practically performed in human mind under broadest reasonable interpretation. For example, a person can recognize the text inside the scoreboard. Claim 19 is not eligible.
Claim 20
In Claim 20 “the processing unit” and associated function is interpreted as invoking 35 U.S.C. 112(f) claim interpretation. Accordingly the algorithm associated with the processing unit is read into the processing unit.
Under broadest reasonable interpretation, claim 20 recites similar steps as claim 1 and therefore recites abstract idea without significantly more. However, under 35 U.S.C. 112(f) claim interpretation, the corresponding algorithm of the processing unit is read into the claim. As described in the specification, the following algorithm is associated with the processing unit: FIG. 2-9, para. [0074]-[0229] (PGPub). These figures and paragraphs describe a specific manner of how the processing unit performs the acts in claim 20 and therefore provide improvements to the functioning of a computer, and apply or use the judicial exception in a meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. Claim 20 is eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 7, 9-12, 16 and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Pan et al. (US Publication 2003/0076448 A1, hereafter Pan), in view of Yokozeki (US Publication 20150146010A1) and Lee et al. (US Publication 2008/0140232 A1, hereafter Lee).
As per claim 1, Pan teaches the invention substantially as claimed including a computer-implemented method (ABSTRACT) to detect a scoreboard within a video frame region, the computer-implemented method comprising:
receiving video streams including a first sequence of video frames and one or more additional video frames (Pan’s scoreboard detection method comprises two parts: training and inference. Training in general is based on the fact that during an entire broadcast video, scoreboards are normally at the same location(s) in different frames, and the remaining areas of frames are dominated by the field or court in most times. Therefore, if a sufficient number of frames containing scoreboard are averaged, then the resulting average image has a clear scoreboard and smeared background that is dominated with the color of the field or the court (i.e., the elements in the field or the court are changing). See para. [0132] & [0136]-[0137]; FIG. 18-19);
analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard (Through training, Pan generates an average image in which the scoreboard region is sufficiently clear and the background is sufficiently blurred. Pan then applies edge detection process to detect edges of the scoreboard, and further obtains a template. See para. [0140]-[0143]; FIG. 20A-20C);
executing a machine-driven pattern recognition procedure to detect, in one or more of the additional video frames, the video frame region that depicts the scoreboard (After generating a scoreboard region template, Pan performs inference for detecting the video frame region that depicts the scoreboard; See para. [0140]-[0145]; FIG. 20A-20C, 21).
Pan further teaches taking an action upon successfully detecting scoreboards in video frames (para. [0135]; claims 35-36), but does not teach the rest limitations.
Yokozeki discloses a face detection method (Abstract; FIG. 2A-2B). Specifically, Yokozeki determines a face area whose degree of reliability is greater than or equal to the threshold value and the number of detection times or detection duration has reached the determination criterion to be a reliable face area (para. [0053]). That is to say, Yokozeki uses two criteria: one is the number of detection times, i.e., the number of a same object detected in a plurality of frames, and the other is the degree of reliability. Only when the two criteria satisfy respective thresholds, the detected face is regarded as reliable (accurate). See FIG. 5A-5C and para. [0058]-[0064] for an example; FIG. 6A-6C and para. [0072]-[0075] for another example; para. [0102] for another embodiment in which a threshold value of the degree of reliability and a threshold value of the number of the detection times are provided for the entire image. Therefore Yokozeki teaches:
based on face detection results (para. [0060]), determining whether at least a number of video frames, from among the video streams, each depict the face within the video frame region (FIG. 2A-2B; FIG. 5A-5C; FIG. 6A-6C; para. [0053], [0066]-[0070], [0074]-[0075]); and
responsive to the determination being that at least the number of video frames exceeds a predetermined number, from among the video streams, each depict the scoreboard within the video frame region, (i) determining that the detected video frame region depicts the scoreboard was accurate (para. [0053]-[0054], [0058]-[0070]); and (ii) responsive to the determining that the detected video frame region depicts the scoreboard was accurate, taking an action (FIG. 9; FIG. 11).
Taking the combined teachings of Pan and Yokozeki as a whole, it would have been obvious for a person with ordinary skill in the art before the effective filing date of the claimed invention to consider: based on results of the machine-driven pattern recognition procedure, determining whether at least a number of video frames, from among the video streams, each depict the scoreboard within the video frame region; and responsive to the determination being that at least the number of video frames exceeds a predetermined number, from among the video streams, each depict the scoreboard within the video frame region, (i) determining that the detected video frame region depicts the scoreboard was accurate and (ii) responsive to the determining that the detected video frame region depicts the scoreboard was accurate, taking an action. The motivation for doing so is to improve object detection reliability and accuracy as recognized by Yokozeki (para. [0085], [0100]).
Pan in view of Yokozeki does not further teach “extracting a sport score from the detected scoreboard region from the video frame region of a video frame of the video streams”.
Lee in the same field of endeavor discloses a method of indexing final pitching shots for each batter in a video recording of a baseball game (Abstract). Specifically, Lee teaches extracting a sport score from a detected scoreboard region from a video frame (See FIG. 6-8 and para. [0039] for locating a scoreboard in video frames, and FIG. 9 and para. [0040] for extracting a sport score).
It would have been obvious to one of ordinary skill in the art, before the effective filing date of this application, to combine the teachings of Lee with Pan and Yokozeki to consider extracting a sport score from the detected scoreboard region. By taking such an action, one can index different parts of a game and generate highlights efficiently as recognized by Lee (para. [0005]).
As per claim 2, dependent upon claim 1, Pan in view of Yokozeki and Lee teaches that receiving the video streams comprises receiving a real-time broadcast feed including the video streams, and wherein automatically detecting the video frame region comprises automatically detecting the video frame region while receiving the real-time broadcast feed (Pan: FIG. 17A-17C; para. [0139]).
As per claim 3, dependent upon claim 1, Pan in view of Yokozeki and Lee teaches the video streams depict a sport-related event (Pan: FIG. 17A-17C), the computer-implemented method further comprising:
determining the sport-related event depicted by the video streams (Pan: FIG. 1; para. [0121]);
based on the determined sport-related event, determining a quantity of video frames to use as basis for automatically detecting the video frame region; and
selecting from among the video streams, the first sequence based on the selected sequence including the determined quantity of video frames (Pan: para. [0136] & [0139]).
As per claim 7, dependent upon claim 1, Pan in view of Yokozeki and Lee further teaches:
wherein the video frames of the first sequence depict image elements including (i) scoreboard image elements that are unchanging across the video frames of the first sequence and (ii) other image elements that change across the video frames of the first sequence (Pan para. [0136]:”Training in general is based on the fact that during an entire broadcast video, scoreboards are normally at the same location(s) in different frames, and the remaining areas of frames are dominated by the field or court in most times”),
wherein analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard is based on detecting a set of edges (Pan para. [0140]-[0141]; FIG. 20A-20C), wherein detecting the video frame region comprises:
based on the edges of the set, engaging in a filtering process to determine actual edges of the scoreboard image elements, wherein the filtering process comprises (i) recognizing edges, from among the edges of the set, based on the recognized edges having a predetermined characteristic and (ii) deeming the recognized edges to be the actual edges of the scoreboard image elements; and
detecting, based on the determined actual edges of the scoreboard image elements, the video frame region that depicts the scoreboard (Pan applies Prewitt edge operator on the resulting average image to detect a set of edges which includes the scoreboard region and non-scoreboard region, as shown in FIG. 20A. Pan then projects the 2-D edge images horizontally and vertically, and obtains two 1-D distributions, as shown in FIGS. 20B and 20C. FIGS. 20B and 20C show the actual edge positions of the scoreboard, obtaining a template of the scoreboard. Pan then detects the video frame region that depicts the scoreboard by inferencing. See para. [0140]-[0145]).
As per claim 9, dependent upon claim 1, Pan in view of Yokozeki and Lee further teaches:
wherein analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard is based on detecting a set of edges (Pan para. [0140]-[0141]; FIG. 20A),
wherein the computer-implemented method comprises accessing region-information indicating a predefined frame region that is not a candidate for depicting the scoreboard,
wherein analyzing the video frames comprises, based on the region-information, excluding, from the analyzing of the video frames, detection of edges in the predefined frame region of the video frames of the first sequence, and
wherein detecting the set of edges comprises detecting the set of edges within one or more frame regions other than the predefined frame region (Pan para. [0125] & [0131] indicates possible locations of scoreboard. Pan further uses edge detection and projection techniques to find the actual boundaries of the scoreboard. The other region than the scoreboard is considered the predefined frame region. As analyzed above in rejections applied to claim 7, Pan further excludes the edges outside the scoreboard region, i.e., the predefined frame region.)
Claim 10, an independent medium claim, recites similar steps as claim 1. Therefore the limitation in claim 10 is mapped to Pan in view of Yokozeki and Lee in the same manner as applied in claim 1. Yokozeki further teaches computer system elements, such as computer-readable storage medium and computer (FIG. 1; para. [0121]). It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Pan to include Yokozeki’s computer system elements in order to implement a method by a computer of a system or apparatus (Yokozeki para. [0121]).
Claim 11, dependent upon claim 10, is rejected as applied above to claim 2.
Claim 12, dependent upon claim 10, is rejected as applied above to claim 3.
Claim 16, dependent upon claim 10, is rejected as applied above to claim 7.
Claim 18, dependent upon claim 10, is rejected as applied above to claim 9.
As per claim 19, dependent upon claim 10, Pan in view of Yokozeki and Lee further teaches:
carrying out, by the computing system, an action based on the detecting of the video frame region that depicts the scoreboard (Pan para. [0135]; claims 35-36).
Claims 4, 8, 13 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Pan et al. (US Publication 2003/0076448 A1, hereafter Pan), in view of Yokozeki (US Publication 20150146010A1) and Lee et al. (US Publication 2008/0140232 A1, hereafter Lee), as applied above to claims 1 and 10 respectively, and further in view of Kesavadas et al. (US Publication 2013/0288214 A1, hereafter Kesavadas).
As per claim 4, Pan in view of Yokozeki and Lee teaches using Prewitt edge operator to detect a set of edges, and wherein detecting the video frame region is based on the detected set of edges (Pan FIG. 20; para. [0140]-[0142]), but does not teach canny edge detection.
Kesavadas is evidenced that canny edge detection is well-known and practiced (para. [0076]).
Taking the combined teachings of Pan, Yokozeki, Lee and Kesavadas as a whole, it would have been obvious for a person with ordinary skill in the art before the effective filing date of the claimed invention to consider using canny edge detection in order to provide an alternative edge detection method as mentioned by Kesavadas (para. [0140]).
As per claim 8, dependent upon claim 1, Pan in view of Yokozeki, Lee and Kesavadas teaches wherein analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard is based on detecting a set of edges (Pan para. [0140]-[0141]; FIG. 20A-20C), wherein detecting the video frame region comprises:
engaging in a morphological-dilatation process to identify a substantially uniform region based on the edges of the set (Kesavadas: para. [0068]); and
based on the substantially uniform region, determining the video frame region that depicts the scoreboard (Pan applies Prewitt edge operator on the resulting average image to detect a set of edges which includes the scoreboard region and non-scoreboard region, as shown in FIG. 20A. Pan then projects the 2-D edge images horizontally and vertically, and obtains two 1-D distributions, as shown in FIGS. 20B and 20C. FIGS. 20B and 20C show the actual edge positions of the scoreboard, obtaining a template of the scoreboard. Pan then detects the video frame region that depicts the scoreboard by inferencing. See para. [0140]-[0145]).
Claim 13, dependent upon claim 10, is rejected as applied above to claim 4.
Claim 17, dependent upon claim 10, is rejected as applied above to claim 8.
Claims 5-6 and 14-15 are rejected under 35 U.S.C. 103 as being unpatentable over Pan et al. (US Publication 2003/0076448 A1, hereafter Pan), in view of Yokozeki (US Publication 2015/0146010A1) and Lee et al. (US Publication 2008/0140232 A1, hereafter Lee), as applied above to claims 1 and 10 respectively, and further in view of Komatsu (US Patent 6,008,866 A).
As per claim 5, Pan in view of Yokozeki and Lee teaches analyzing the video frames of the first sequence to detect the video frame region that depicts the scoreboard is based on detecting a set of edges (Pan para. [0140]-[0141]; FIG. 20A-20C), but does not teach the rest limitations recited in claim 5.
Komatsu discloses system and method for edge detection in moving pictures in a sequence (ABSTRACT). Specifically, Komatsu teaches:
the sequence includes a first video frame and a second video frame, wherein the set of edges comprises first edges and second edges (FIG. #1 & #3),
wherein detecting the set comprises detecting the first edges in the first video frame and detecting the second edges in the second video frame (FIG. 1 #6-#9; col. 5 lines 26-49), and
wherein detecting the video frame region that depicts the scoreboard comprises (i) applying a logical conjunction on the detected first and second edges to produce an output that indicates common edges detected in both the first video frame and the second video frame, and (ii) deeming the common edges to be the edges of a subset of edges, wherein detecting the video frame region is based on the subset of edges (FIG. 1 #10; FIG. 3 section b; col. 5 lines 51-58).
Taking the combined teachings of Pan, Yokozeki, Lee and Komatsu as a whole, it would have been obvious for a person of ordinary skill in the art before the effective filing date of the claimed invention to consider detecting edges in a manner as recited in claim 5. Doing so would accurately determine static edges in a sequence of video frames as suggested by Komatsu (col. 2 ln 62-65).
As per claim 6, dependent upon claim 5, Pan in view of Yokozeki, Lee and Komatsu teaches:
wherein analyzing the video frames generates (i) a first binary representation of the first edges detected in the first video frame and (ii) a second binary representation of the second edges detected in the second video frame,
wherein applying the logical conjunction on the detected first and second edges comprises applying the logical conjunction on the first and second binary representations, and
wherein the output comprises a third binary representation of the common edges (Pan teaches generating binary images (FIG. 20A; para. [0141]). Note logical operation is operable on two images. Therefore the combination of Pan, Yokozeki, Lee and Komatsu renders obvious generating a first binary image and a second binary image, applying the logical conjunction on the first and second binary representations, and outputting a third binary representation of the common edges (The third binary image is the result of the logical conjunction on the first and second binary representations). See Komatsu FIG. 1 #10; FIG. 3 section b; col. 5 lines 51-58). It would have been obvious to a person with ordinary skill in the art before the effective filing date of the claimed invention to modify the teaching of Pan to incorporate the teaching of Komatsu to generate (i) a first binary representation of the first edges detected in the first video frame and (ii) a second binary representation of the second edges detected in the second video frame, and apply the logical conjunction on the first and second binary representations, and output a third binary representation of the common edges. Doing so would detect edge motion as recognized by Komatsu (col. 5 ln 51-58).
Claim 14, dependent upon claim 10, is rejected as applied above to claim 5.
Claim 15, dependent upon claim 14, is rejected as applied above to claim 6.
Allowable Subject Matter
Claim 20 would be allowable should the Double Patenting rejection applied above is overcome.
The following is Examiner’s reasons for identification of allowable subject matter.
In independent claim 20, “a processing unit” and associated function is interpreted as invoking 35 USC 112 (f) claim interpretation. Since the unit is computer-implemented element, the corresponding algorithm in the specification is being read into the element interpreted under 112(f) as identified above. Specifically, at least FIG. 2-9, para. [0074]-[0229] (PGPub) of the instant specification contains allowable subject matter when taken in combination with the entirety of the claim.
The following prior art is considered closest to the current application.
Under broadest reasonable interpretation, the combination of Pan et al. (US Publication 2003/0076448 A1, hereafter Pan), Yokozeki (US Publication 20150146010A1) and Lee et al. (US Publication 2008/0140232 A1, hereafter Lee) teaches method claim 1 and medium claim 10 as applied above. Claim 20 recites similar steps as in claim 1/10.
Prior art, either applied alone, or in combination with, fails to teach or suggest all the limitations as recited in claim 20 in view of 35 USC 112(f) claim interpretation.
Conclusion
Prior art searched but not cited is recorded in PTO-892.
Additional prior JEONG et al. (US 20110052061 A1) discloses a method for detecting a scoreboard. The method includes: detecting first candidate areas that are presumed to include important information in a plurality of moving picture frames by using stop edge information, which is edge information overlapped at a same position throughout the plurality of moving picture frames, from among edge information in at least two received moving picture frames; determining second candidate areas by performing grouping on the stop edge information according to a position of the stop edge information in the first candidate areas; analyzing the second candidate areas determined in the at least two moving picture frames; and detecting important information areas from each of the at least two moving picture frames based on the analysis. See Abstract, FIG. 5-6, para. [0080]-[0099].
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUEMEI G CHEN whose telephone number is (571)270-3480. The examiner can normally be reached Monday-Friday 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached on (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XUEMEI G CHEN/Primary Examiner, Art Unit 2661
