DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 are pending in the application.
Claim Objections
Claim 8 the preamble recites “cause at least one processor to perform …”, while the body of the claim recites “by a/the computing system”. What is the relationship between the “at least one processor” and the “computing system”? Clarification is required. For 112f analysis and prior art consideration, Examiner considers the at least one processor overrides the computing system.
Claim 15 recites a computing device comprises “at least one processor” and “cause the at least one processor to perform …”, while the body of the claim recites “by a/the computing system”. What is the relationship between the “at least one processor”/ “computing device” and the “computing system”? Clarification is required. For 112f analysis and prior art consideration, Examiner considers the at least one processor overrides the computing system.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the claims at issue are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. Please visit http://www.uspto.gov/forms/. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claims 1, 8 and 15 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 8 and 13 of U.S. Patent No. US 12,217,494 B2 respectively. Although the claims at issue are not identical, they are not patentably distinct from each other because of the following reasons.
Listed in the following tables is a limitation-to-limitation comparison of the examined claims and the conflicting claims.
Application being examined 18/999,638 (hereafter ‘638 application)
Conflicting patent 12,217,494 (hereafter ‘494 patent)
1. A method comprising:
determining, by a computing system, a first blur score for a first frame of a video based on a discrete cosine transform (DCT) of pixel intensity values of the first frame by:
determining a first matrix of DCT coefficients based on the first frame;
determining a second matrix of DCT coefficients based on a transposition of the first matrix of DCT coefficients; and
determining the first blur score for the first frame based on the second matrix of DCT coefficients;
determining, by the computing system, a second blur score for a second frame of the video based on a DCT of pixel intensity values of the second frame by:
determining a third matrix of DCT coefficients based on the second frame;
determining a fourth matrix of DCT coefficients based on a transposition of the third matrix of DCT coefficients; and
determining the second blur score for the second frame based on the fourth matrix of DCT coefficients;
determining, by the computing system, a blur delta that quantifies a difference between a level of blurriness of the first frame of the video represented by the first blur score and a level of blurriness of the second frame of the video represented by the second blur score, wherein the second frame is subsequent to and adjacent to the first frame;
determining, by the computing system, a contrast delta that quantifies a difference between a contrast of the first frame and a contrast of the second frame;
determining, by the computing system, a fingerprint distance between a first image fingerprint of the first frame and a second image fingerprint of the second frame;
determining, by the computing system, a keyframe score using the blur delta, the contrast delta, and the fingerprint distance;
based on the keyframe score, determining, by the computing system, that the second frame is a keyframe; and
outputting, by the computing system, data indicating that the second frame is a keyframe.
1. A method comprising:
determining, by a computing system, a first blur score for a first frame of a video based on a discrete cosine transform (DCT) of pixel intensity values of the first frame by:
converting the first frame to grayscale;
downscaling the grayscale first frame;
calculating a first DCT of the grayscale, downscaled first frame to determine a first matrix of DCT coefficients;
transposing the first matrix of DCT coefficients;
calculating a second DCT of the transposed first matrix of DCT coefficients to determine a second matrix of DCT coefficients; and
calculating the first blur score for the first frame by subtracting a sum of the absolute values of the upper-left quarter of the second matrix of DCT coefficients from a sum of the absolute values of the coefficients of the second matrix of DCT coefficients;
determining, by the computing system, a second blur score for a second frame of the video based on a DCT of pixel intensity values of the second frame by:
converting the second frame to grayscale;
downscaling the grayscale second frame;
calculating a third DCT of the grayscale, downscaled second frame to determine a third matrix of DCT coefficients;
transposing the third matrix of DCT coefficients;
calculating a fourth DCT of the transposed third matrix of DCT coefficients to determine a fourth matrix of DCT coefficients; and
calculating the second blur score for the second frame by subtracting a sum of the absolute values of the upper-left quarter of the fourth matrix of DCT coefficients from a sum of the absolute values of the coefficients of the fourth matrix of DCT coefficients;
determining, by the computing system, a blur delta that quantifies a difference between a level of blurriness of the first frame of the video represented by the first blur score and a level of blurriness of the second frame of the video represented by the second blur score, wherein the second frame is subsequent to and adjacent to the first frame;
determining, by the computing system, a contrast delta that quantifies a difference between a contrast of the first frame and a contrast of the second frame;
determining, by the computing system, a fingerprint distance between a first image fingerprint of the first frame and a second image fingerprint of the second frame;
determining, by the computing system, a keyframe score using the blur delta, the contrast delta, and the fingerprint distance;
based on the keyframe score, determining, by the computing system, that the second frame is a keyframe; and
outputting, by the computing system, data indicating that the second frame is a keyframe.
Application being examined 18/999,638 (hereafter ‘638 application)
Conflicting patent 12,217,494 (hereafter ‘494 patent)
8. A tangible, non-transitory computer readable medium comprising instructions that, when executed, cause at least one processor to perform a set of operations comprising:
determining, by a computing system, a first blur score for a first frame of a video based on a discrete cosine transform (DCT) of pixel intensity values of the first frame by:
determining a first matrix of DCT coefficients based on the first frame;
determining a second matrix of DCT coefficients based on a transposition of the first matrix of DCT coefficients; and
determining the first blur score for the first frame based on the second matrix of DCT coefficients;
determining, by the computing system, a second blur score for a second frame of the video based on a DCT of pixel intensity values of the second frame by:
determining a third matrix of DCT coefficients based on the second frame;
determining a fourth matrix of DCT coefficients based on a transposition of the third matrix of DCT coefficients; and
determining the second blur score for the second frame based on the fourth matrix of DCT coefficients;
determining, by the computing system, a blur delta that quantifies a difference between a level of blurriness of the first frame of the video represented by the first blur score and a level of blurriness of the second frame of the video represented by the second blur score, wherein the second frame is subsequent to and adjacent to the first frame;
determining, by the computing system, a contrast delta that quantifies a difference between a contrast of the first frame and a contrast of the second frame;
determining, by the computing system, a fingerprint distance between a first image fingerprint of the first frame and a second image fingerprint of the second frame;
determining, by the computing system, a keyframe score using the blur delta, the contrast delta, and the fingerprint distance;
based on the keyframe score, determining, by the computing system, that the second frame is a keyframe; and
outputting, by the computing system, data indicating that the second frame is a keyframe.
8. A non-transitory computer-readable medium having stored thereon program instructions that upon execution by a processor, cause performance of a set of acts comprising:
determining a first blur score for a first frame of a video based on a discrete cosine transform (DCT) of pixel intensity values of the first frame by:
converting the first frame to grayscale;
downscaling the grayscale first frame;
calculating a first DCT of the grayscale, downscaled first frame to determine a first matrix of DCT coefficients;
transposing the first matrix of DCT coefficients;
calculating a second DCT of the transposed first matrix of DCT coefficients to determine a second matrix of DCT coefficients; and
calculating the first blur score for the first frame by subtracting a sum of the absolute values of the upper-left quarter of the second matrix of DCT coefficients from a sum of the absolute values of the coefficients of the second matrix of DCT coefficients;
determining a second blur score for a second frame of the video based on a DCT of pixel intensity values of the second frame by:
converting the second frame to grayscale;
downscaling the grayscale second frame;
calculating a third DCT of the grayscale, downscaled second frame to determine a third matrix of DCT coefficients;
transposing the third matrix of DCT coefficients;
calculating a fourth DCT of the transposed third matrix of DCT coefficients to determine a fourth matrix of DCT coefficients; and
calculating the second blur score for the second frame by subtracting a sum of the absolute values of the upper-left quarter of the fourth matrix of DCT coefficients from a sum of the absolute values of the coefficients of the fourth matrix of DCT coefficients;
determining a blur delta that quantifies a difference between a level of blurriness of the first frame of the video represented by the first blur score and a level of blurriness of the second frame of the video represented by the second blur score, wherein the second frame is subsequent to and adjacent to the first frame;
determining a contrast delta that quantifies a difference between a contrast of the first frame and a contrast of the second frame;
determining a fingerprint distance between a first image fingerprint of the first frame and a second image fingerprint of the second frame;
determining a keyframe score using the blur delta, the contrast delta, and the fingerprint distance;
based on the keyframe score, determining that the second frame is a keyframe; and
outputting data indicating that the second frame is a keyframe.
Application being examined 18/999,638 (hereafter ‘638 application)
Conflicting patent 12,217,494 (hereafter ‘494 patent)
15. A computing device comprising:
at least one processor; and tangible, non-transitory computer readable medium comprising instructions that, when executed, cause the at least one processor to perform a set of operations comprising:
determining, by a computing system, a first blur score for a first frame of a video based on a discrete cosine transform (DCT) of pixel intensity values of the first frame by:
determining a first matrix of DCT coefficients based on the first frame;
determining a second matrix of DCT coefficients based on a transposition of the first matrix of DCT coefficients; and
determining the first blur score for the first frame based on the second matrix of DCT coefficients;
determining, by the computing system, a second blur score for a second frame of the video based on a DCT of pixel intensity values of the second frame by:
determining a third matrix of DCT coefficients based on the second frame;
determining a fourth matrix of DCT coefficients based on a transposition of the third matrix of DCT coefficients; and
determining the second blur score for the second frame based on the fourth matrix of DCT coefficients;
determining, by the computing system, a blur delta that quantifies a difference between a level of blurriness of the first frame of the video represented by the first blur score and a level of blurriness of the second frame of the video represented by the second blur score, wherein the second frame is subsequent to and adjacent to the first frame;
determining, by the computing system, a contrast delta that quantifies a difference between a contrast of the first frame and a contrast of the second frame;
determining, by the computing system, a fingerprint distance between a first image fingerprint of the first frame and a second image fingerprint of the second frame;
determining, by the computing system, a keyframe score using the blur delta, the contrast delta, and the fingerprint distance;
based on the keyframe score, determining, by the computing system, that the second frame is a keyframe; and
outputting, by the computing system, data indicating that the second frame is a keyframe.
13. A computing system comprising:
a processor; and
a non-transitory computer-readable medium, having stored thereon program instructions that upon execution by the processor, cause the computing system to perform a set of acts comprising:
determining a first blur score for a first frame of a video based on a discrete cosine transform (DCT) of pixel intensity values of the first frame by:
converting the first frame to grayscale;
downscaling the grayscale first frame;
calculating a first DCT of the grayscale, downscaled first frame to determine a first matrix of DCT coefficients;
transposing the first matrix of DCT coefficients;
calculating a second DCT of the transposed first matrix of DCT coefficients to determine a second matrix of DCT coefficients; and
calculating the first blur score for the first frame by subtracting a sum of the absolute values of the upper-left quarter of the second matrix of DCT coefficients from a sum of the absolute values of the coefficients of the second matrix of DCT coefficients;
determining a second blur score for a second frame of the video based on a DCT of pixel intensity values of the second frame by:
converting the second frame to grayscale;
downscaling the grayscale second frame;
calculating a third DCT of the grayscale, downscaled second frame to determine a third matrix of DCT coefficients;
transposing the third matrix of DCT coefficients;
calculating a fourth DCT of the transposed third matrix of DCT coefficients to determine a fourth matrix of DCT coefficients; and
calculating the second blur score for the second frame by subtracting a sum of the absolute values of the upper-left quarter of the fourth matrix of DCT coefficients from a sum of the absolute values of the coefficients of the fourth matrix of DCT coefficients;
determining a blur delta that quantifies a difference between a level of blurriness of the first frame of the video represented by the first blur score and a level of blurriness of the second frame of the video represented by the first blur score, wherein the second frame is subsequent to and adjacent to the first frame;
determining a contrast delta that quantifies a difference between a contrast of the first frame and a contrast of the second frame;
determining a fingerprint distance between a first image fingerprint of the first frame and a second image fingerprint of the second frame;
determining a keyframe score using the blur delta, the contrast delta, and the fingerprint distance;
based on the keyframe score, determining that the second frame is a keyframe; and
outputting data indicating that the second frame is a keyframe.
Therefore claim 1/8/13 of the ‘494 patent teaches every limitation in claim 1/8/15 of the ‘638 application.
CLAIM INTERPRETATION
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. The following table lists the occurrences that use means and corresponding structure and associated algorithm.
Claim no.
112(f) elements
Corresponding structure (PGPub)
Associated algorithm (PGPub)
1
determining/outputting, by a/the computing system
FIG. 1 #102 “Processor”
FIG. 3-5; para. [0036]-[0071]
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Allowable Subject Matter
Claims 1-20 would be allowable should the Double Patenting rejection presented in this OA is overcome.
The following is Examiner’s reasons for identification of allowable subject matter.
In independent claims 1, 8 and 15, the following limitations are recited:
determining, a first blur score for a first frame of a video based on a discrete cosine transform (DCT) of pixel intensity values of the first frame by:
determining a first matrix of DCT coefficients based on the first frame;
determining a second matrix of DCT coefficients based on a transposition of the first matrix of DCT coefficients; and
determining the first blur score for the first frame based on the second matrix of DCT coefficients;
determining, second blur score for a second frame of the video based on a DCT of pixel intensity values of the second frame by:
determining a third matrix of DCT coefficients based on the second frame;
determining a fourth matrix of DCT coefficients based on a transposition of the third matrix of DCT coefficients; and
determining the second blur score for the second frame based on the fourth matrix of DCT coefficients;
determining, a blur delta that quantifies a difference between a level of blurriness of the first frame of the video represented by the first blur score and a level of blurriness of the second frame of the video represented by the second blur score, wherein the second frame is subsequent to and adjacent to the first frame;
determining, contrast delta that quantifies a difference between a contrast of the first frame and a contrast of the second frame;
determining, a fingerprint distance between a first image fingerprint of the first frame and a second image fingerprint of the second frame;
determining, a keyframe score using the blur delta, the contrast delta, and the fingerprint distance;
based on the keyframe score, determining, that the second frame is a keyframe; and
outputting, data indicating that the second frame is a keyframe.
The following prior art is considered the closest to the current application.
Zhang (US 2007/0263128 A1) teaches a method (Abstract) comprising:
determining, a level of blurriness of a first frame of a video and a level of blurriness of a second frame of the video (Zhang segments a video into a set of shots each having a set of video frames. Zhang further calculates blurriness of each frame in a shot, including a first frame and a second frame. See Abstract, FIG. 4 #72, para. [0020]-[0021], [0029]);
determining, a contrast of the first frame and a contrast of the second frame (Zhang further calculates contrast of each frame in a shot, including a first frame and a second frame. See FIG. 4 #76, para. [0030]);
determining, a first image fingerprint of the first frame and a second image fingerprint of the second frame (Zhang further calculates a fingerprint (a characteristic) of each frame in a shot, including a first frame and a second frame. See FIG. 4 #70, para. [0027]);
determining, a quality score of each frame using the level of blurriness, the contrast, and the fingerprint (para. [0046]);
based on the quality score, determining, that the second frame is a keyframe (Zhang further teaches ranking quality scores of video frames and selecting the video frame with the best quality score as the key-frame (para. [0046], para. [0050]). Note the ranking and selecting process includes comparison of the quality scores between two frames); and
outputting, data indicating that the second frame is a keyframe (FIG. 5-7).
Zhang teaches calculating 3 components of a quality score for each frame in a shot including a first frame and a second frame, but does not teach calculating difference between corresponding components, and does not specify that the second frame is subsequent to and adjacent to the first frame.
Moriya et al. (US 2005/0149557 A1, hereafter Moriya) discloses a method for detecting a scene change point (para. [0069]-[0077]). Moriya further mentions that such a change point can be used for selecting a representative frame for a scene (para. [0050]). Specifically, histograms for each frame and each color channel are calculated. The difference of histograms between two adjacent frames, for example, a first frame i-1 and a second frame i (frame i being subsequent to frame i-1 and being adjacent to frame i-1), is calculated as a representative frame score for scene change point detection (para. [0070]-[0074] eqn. 1). When the score is greater than a threshold, it is detected there is a scene change point (para. [0077]). Note the difference of histograms represents pixel value differences between two frames. The difference is indicative of sharpness difference, contrast difference or fingerprint difference.
Javaran et al. (Javaran TA, Hassanpour H, Abolghasemi V. A noise-immune no-reference metric for estimating blurriness value of an image. Signal Processing: Image Communication. 2016 Sep 1;47:218-28) discloses a method for estimating blurriness of an image by calculating a no-reference blur metric (Abstract). Specifically, lp norm ratio between DCT coefficients of a given image and those of the blurred version of the given image is calculated as a blur metric. The blurred version of the given image is obtained by using a low pass filter on the given image. By doing so, the noise effect is mitigated in via discarding the higher order DCT coefficients. See page 220-221 section 2.1. The blur metric.
Prior art, either applied alone or in combination with, fails to disclose or suggest the claimed method of calculating the keyframe score using the blur delta, the contrast delta, and the fingerprint distance. Additionally, for claim 1, the corresponding algorithm associated with the “computing system” is read into the claim in view of the 112f claim interpretation as identified above.
Conclusion
Prior art searched but not cited is recorded in PTO-892.
Additional prior art Feng (US 20040120598 A1) discloses a method of detecting and correcting blur in a digital image stored in a digital file as a sequence of DCT coefficients arranged in a plurality of blocks representing a grid of frequency components, each said block representing a portion of said image, said method comprising: (a) reading a plurality of DCT coefficients from said sequence of DCT coefficients, said plurality of DCT coefficients comprising at least two DCT coefficients from each of said plurality of blocks; (b) calculating from said plurality of DCT coefficients a first blur indicator that quantifies the amount of blur in the block having the least amount of blur among all of said plurality of blocks; (c) calculating from said plurality of DCT coefficients a second blur indicator that quantifies the amount of blur throughout said digital image; and (d) selectively applying a filter to said digital image based upon the value of said first blur indicator and said second blur indicator. See FIG. 6-7, para. [0025], claim 1.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to XUEMEI G CHEN whose telephone number is (571)270-3480. The examiner can normally be reached Monday-Friday 9am-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John M Villecco can be reached at (571) 272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/XUEMEI G CHEN/Primary Examiner, Art Unit 2661
Prosecution Insights