DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 4-5 and 10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by U.S. Patent No. 8,270,476 to Schmit et al. (hereinafter Schmit).
Regarding independent claim 1, Schmit discloses A bit rate control device (column 5, line 54, “ The rate controller component 604 dynamically adjusts encoder parameters to achieve a target bitrate specified by a bitrate parameter 606. The rate controller 604 allocates a budget of bits to each region, individual picture, group of pictures and/or sub-picture in a video sequence. ”), comprising:
a confidence calculation circuit, calculating a texture confidence (column 5, line 59, “The bitrate algorithm essentially uses the activity map data provided by the pre-processor to determine the activity of the face region relative to the average activity and allocate bits in accordance with what is important.” The activity map is read as the texture confidence), a person confidence (column 5, line 18, “The activity map data 205 is combined in combinatorial circuit 212 with face data 207 provided by the face detector module 202. In one embodiment, the face data 207 comprises a value corresponding to the certainty of the face detection. This may be a data value along a defined scale, such as 0-1, or any similar range. The combinatorial circuit 212 may be implemented as a logic module that combines the activity and face data through any appropriate combination algorithm.”) and a motion confidence according to an image (column 6, line 1, “In one embodiment, face detection method may utilize the motion estimator 206 stage of the encoder pipeline 200. For each macroblock in a current frame, the motion estimator 206 attempts to find a region in a previously encoded frame (reference frame) that is a close match. The spatial offset between the current block and selected block from the reference frame is a motion vector. The encoder 206 computes the pixel-by-pixel difference between the selected block from the reference frame and the current block and transmits a resultant prediction error along with the motion vector. For this embodiment, the output from motion estimator 206 is provided as an input to combinatorial circuit 212. The motion vector data is then combined along with the activity map and face data to provide bitrate allocation parameters to bitrate control module 208.”);
a blend calculation circuit, determining, according to a mode selection signal (column 6, line 61, “ An enhanced lossless macroblock representation mode allows perfect representation of specific regions while ordinarily using substantially fewer bits than the PCM mode”), whether to select one of the person confidence and the motion confidence for blend calculation with the texture confidence to generate a blend parameter (column 5, line 59, “ The bitrate algorithm essentially uses the activity map data provided by the pre-processor to determine the activity of the face region relative to the average activity and allocate bits in accordance with what is important. In one embodiment, the bitrate parameter 606 is provided by the output of combinatorial circuit 212 to control the quantization and the fine grained quality of the face object detected by face detector 202. This allows enhanced detail to be provided to the face image.”); and
a bit distribution circuit, generating a bit distribution parameter according to the blend parameter (column 5, line 54, “ The rate controller component 604 dynamically adjusts encoder parameters to achieve a target bitrate specified by a bitrate parameter 606. The rate controller 604 allocates a budget of bits to each region, individual picture, group of pictures and/or sub-picture in a video sequence. ”).
Regarding dependent claim 4, the rejection of claim 1 is incorporated herein. Additionally, Schmit further discloses wherein the blend calculation circuit comprises:
a texture blend calculator, setting the texture confidence as the blend parameter in response to the mode selection signal (column 5, line 59, “The bitrate algorithm essentially uses the activity map data provided by the pre-processor to determine the activity of the face region relative to the average activity and allocate bits in accordance with what is important;” this is read as a texture only mode in that the activity-map-driven baseline is used when neither person or motion emphases is selected);
a person blend calculator, configured to perform the blend calculation on the person confidence and the texture confidence to generate the blend parameter (column 5, line 63, “ the bitrate parameter 606 is provided by the output of combinatorial circuit 212 to control the quantization and the fine grained quality of the face object detected by face detector 202. This allows enhanced detail to be provided to the face image.”); and
a motion blend calculator, configured to perform the blend calculation on the motion confidence and the texture confidence to generate the blend parameter (column 6, line 1, “In one embodiment, face detection method may utilize the motion estimator 206 stage of the encoder pipeline 200. For each macroblock in a current frame, the motion estimator 206 attempts to find a region in a previously encoded frame (reference frame) that is a close match. The spatial offset between the current block and selected block from the reference frame is a motion vector. The encoder 206 computes the pixel-by-pixel difference between the selected block from the reference frame and the current block and transmits a resultant prediction error along with the motion vector. For this embodiment, the output from motion estimator 206 is provided as an input to combinatorial circuit 212. The motion vector data is then combined along with the activity map and face data to provide bitrate allocation parameters to bitrate control module 208.”).
Regarding dependent claim 5, the rejection of claim 4 is incorporated herein. Additionally, Schmit in the combination further discloses wherein the blend calculation circuit further comprises: a selector, transmitting the texture confidence to one of the texture blend calculator, the person blend calculator and the motion blend calculator according to the mode selection signal (column 5, line 54, “ The rate controller component 604 dynamically adjusts encoder parameters to achieve a target bitrate specified by a bitrate parameter 606. The rate controller 604 allocates a budget of bits to each region, individual picture, group of pictures and/or sub-picture in a video sequence. ”).
Regarding independent claim 10, the rejection of claim 1 applies directly. Additionally, Schmit in the combination further discloses A bit rate control method, applied to a bit rate control device (column 5, line 54, “ The rate controller component 604 dynamically adjusts encoder parameters to achieve a target bitrate specified by a bitrate parameter 606. The rate controller 604 allocates a budget of bits to each region, individual picture, group of pictures and/or sub-picture in a video sequence. ”), the method comprising:
calculating a texture confidence(column 5, line 59, “The bitrate algorithm essentially uses the activity map data provided by the pre-processor to determine the activity of the face region relative to the average activity and allocate bits in accordance with what is important.” The activity map is read as the texture confidence), a person confidence (column 5, line 18, “The activity map data 205 is combined in combinatorial circuit 212 with face data 207 provided by the face detector module 202. In one embodiment, the face data 207 comprises a value corresponding to the certainty of the face detection. This may be a data value along a defined scale, such as 0-1, or any similar range. The combinatorial circuit 212 may be implemented as a logic module that combines the activity and face data through any appropriate combination algorithm.”) and a motion confidence according to an image (column 6, line 1, “In one embodiment, face detection method may utilize the motion estimator 206 stage of the encoder pipeline 200. For each macroblock in a current frame, the motion estimator 206 attempts to find a region in a previously encoded frame (reference frame) that is a close match. The spatial offset between the current block and selected block from the reference frame is a motion vector. The encoder 206 computes the pixel-by-pixel difference between the selected block from the reference frame and the current block and transmits a resultant prediction error along with the motion vector. For this embodiment, the output from motion estimator 206 is provided as an input to combinatorial circuit 212. The motion vector data is then combined along with the activity map and face data to provide bitrate allocation parameters to bitrate control module 208.”);
determining, according to a mode selection signal (column 6, line 61, “ An enhanced lossless macroblock representation mode allows perfect representation of specific regions while ordinarily using substantially fewer bits than the PCM mode”), whether to select one of the person confidence and the motion confidence for blend calculation with the texture confidence to generate a blend parameter (column 5, line 59, “ The bitrate algorithm essentially uses the activity map data provided by the pre-processor to determine the activity of the face region relative to the average activity and allocate bits in accordance with what is important. In one embodiment, the bitrate parameter 606 is provided by the output of combinatorial circuit 212 to control the quantization and the fine grained quality of the face object detected by face detector 202. This allows enhanced detail to be provided to the face image.”); and
generating a bit distribution parameter according to the blend parameter (column 5, line 54, “ The rate controller component 604 dynamically adjusts encoder parameters to achieve a target bitrate specified by a bitrate parameter 606. The rate controller 604 allocates a budget of bits to each region, individual picture, group of pictures and/or sub-picture in a video sequence. ”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6 is rejected under 35 U.S.C. 103 as being unpatentable over Schmit as applied to claim 1 above, and further in view of U.S. Patent No. 8,548,049 to Pan et al. (hereinafter Pan) and U.S. Publication No. 2002/0141498 to Martins et al. (hereinafter Martins).
Regarding dependent claim 6, the rejection of claim 1 is incorporated herein. Additionally, Schmit fails to explicitly disclose wherein when the person confidence is greater than a person confidence threshold, the blend calculation circuit generates the blend parameter from the person confidence and the texture confidence according to the mode selection signal; when the motion confidence is greater than a motion confidence threshold, the blend calculation circuit generates the blend parameter from the texture confidence and the motion confidence according to the mode selection signal.
However, Pan discloses wherein when the person confidence is greater than a person confidence threshold, the blend calculation circuit generates the blend parameter from the person confidence and the texture confidence according to the mode selection signal (abstract, “ An encoder section, generates the processed video signal and wherein, when the pattern of interest is detected, a higher quantization is assigned to the region than to portions of the at least one image outside the region.”);
Schmit is directed toward, “Embodiments include a codec for use in a videoconferencing or similar system includes a video encoder pipeline that has a pre-processor component that is optimized to detect faces and compress the facial video data in an optimum manner. The codec has a pre-processing step that analyzes each frame on a per macroblock basis to determine the mathematical activity level per block (abstract).” Pan is directed toward, “A system for encoding a video stream into a processed video signal that includes at least one image, includes a pattern detection module for detecting a pattern of interest in the at least one image and identifying a region that contains the pattern of interest when the pattern of interest is detected (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Schmit and Pan are directed toward similar methods of video analysis. Further, one of ordinary skill in the art before the effective filing date would be easily be aware it would be ideal t o threshold confidence to ensure an object is a person, before applying the person processing to the region. Said differently, if the region was likely not a person (i.e. low confidence) it would be obvious to not apply the person processing in order to not perform inaccurate processing on the image as a whole. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Pan in order to ensure processing for specific regions is only applied to regions that are likely to meet that criteria.
Schmit and Pan fail to explicitly disclose as further recited. However, Martins discloses when the motion confidence is greater than a motion confidence threshold, the blend calculation circuit generates the blend parameter from the texture confidence and the motion confidence according to the mode selection signal (paragraph 0024, “For example, in an image of a person waving a hand, the region in which the hand is moving has lots of activity due to non-compensated motion and to ensure that the spatial quality of the video is maintained, more bits are required by the moving hand than by a stationary hand.”).
As noted above, Schmit and Pan are directed toward similar methods of endeavor of video analysis. Further, Schmit is directed toward, “Embodiments include a codec for use in a videoconferencing or similar system includes a video encoder pipeline that has a pre-processor component that is optimized to detect faces and compress the facial video data in an optimum manner (abstract).” Martins is directed toward, “A source model in combination with an interest structure is provided to generate a quantization value for use in encoding a video signal. The interest structure is generated from a region of interest manually identified by a user viewing the video on an interactive user display or automatically by a system which recognizes the regions of interest automatically (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Schmit and Martins are directed toward similar methods of endeavor of video data analysis and region determination. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand detecting motion across video data can be performed by comparing frame data where differences represent movement. Further, movement often is considered an area of interest in video being that there is some sort of changing feature when motion is present. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to ensure the moving regions can be analyzed, and feature can be preserved in bit rate determination.
Claim(s) 8 rejected under 35 U.S.C. 103 as being unpatentable over Schmit as applied to claim 1 above, and further in view of U.S. Patent No. 8,917,765 to Liu et al. (hereinafter Liu).
Regarding dependent claim 8, the rejection of claim 1 is incorporated herein. Additionally, Schmit fails to explicitly disclose wherein the confidence calculation circuit comprises:
a person confidence calculator, performing skin color determination on the image to generate a skin color region, performing morphological processing on the skin color region to generate a morphological processing result, and sampling the morphological processing result to obtain the person confidence.
However, Liu discloses wherein the confidence calculation circuit comprises:
a person confidence calculator, performing skin color determination on the image to generate a skin color region (column 3, line 57, “can generate detected region 322 based on the detection of pixel color values corresponding to facial features such as skin tones. ”), performing morphological processing on the skin color region to generate a morphological processing result (column 3, line 46, “ Region identification signal generator 150 optionally includes a region cleaning module 324 that generates a clean region 326 based on the detected region 322, such via a morphological operation. Region identification signal generator 150 can further include a region growing module that expands the clean region 326 to generate a region identification signal 330 that identifies the region containing the pattern of interest.”), and sampling the morphological processing result to obtain the person confidence (column 3, line 59, “Region cleaning module can generate a more contiguous region that contains these facial features and region growing module can grow this region to include the surrounding hair and other image portions to ensure that the entire face is included in the region identified by region identification signal 330. ”).
Schmit is directed toward, “Embodiments include a codec for use in a videoconferencing or similar system includes a video encoder pipeline that has a pre-processor component that is optimized to detect faces and compress the facial video data in an optimum manner” and “The activity level calculation is used as a parameter to the bitrate control module of the encoder to control the quantization, and thus the fine grained quality of the output data. An object detection module (e.g., a face detector) is placed in the pre-processing step. The object detection data is then combined with the activity level and object detection certainty value through a combinatorial algorithm comprising a weighted average or normalized multiplication process (abstract).” Liu is directed toward, “A system for encoding a video stream into a processed video signal that includes at least one image, includes a region identification signal generator for detecting a region of interest in the at least one image and generating a region identification signal when the pattern of interest is detected (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Schmit and Liu are directed toward similar methods of endeavor of image analysis and region detection. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would be easily aware when detecting humans, skin and additional facial features can be incredibly helpful, as well as provide detailed information that one would want in a higher bit stream. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Liu in order to ensure image details are preserved where needed using bit allocation.
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over Schmit as applied to claim 1 above, and further in view of Martins
Regarding dependent claim 9, the rejection of claim 1 is incorporated herein. Additionally, Schmit fails to explicitly disclose wherein the confidence calculation circuit comprises:
a motion confidence calculator, sampling a plurality of original-frame macroblocks of an original frame of the image and a plurality of previous-frame macroblocks of a previous frame of the original frame, calculating a plurality of frame differences according to the original-frame macroblocks and the previous-frame macroblocks, and calculating the motion confidence according to the frame differences.
However, Martins discloses wherein the confidence calculation circuit comprises:
a motion confidence calculator, sampling a plurality of original-frame macroblocks of an original frame of the image and a plurality of previous-frame macroblocks of a previous frame of the original frame (paragraph 0023, “ the current and the previous motion compensated frames are partitioned into macroblocks”), calculating a plurality of frame differences according to the original-frame macroblocks and the previous-frame macroblocks (paragraph 0023, “For example, to calculate σi using the sum of the absolute differences after motion compensation, the current and the previous motion compensated frames are partitioned into macroblocks;” paragraph 0023, “For each corresponding pair of macroblocks, a pixel-by-pixel difference is computed and the sum of the absolute value of the differences is computed. If there is no motion between the two frames, or if motion compensation is perfect, then the selected macroblocks are identical and the sum of the absolute differences is zero. If there is a small amount of non-compensated motion, then the sum of the absolute differences is small, and if there is a large amount of non-compensated motion, then the sum of the absolute differences is large. The sum of weighted differences is an approximation to the sum of absolute differences, and in one embodiment, the sum of weighted differences is a faster and more efficient method of calculation than the sum of absolute differences.”), and calculating the motion confidence according to the frame differences (paragraph 0023, “For each corresponding pair of macroblocks, a pixel-by-pixel difference is computed and the sum of the absolute value of the differences is computed. If there is no motion between the two frames, or if motion compensation is perfect, then the selected macroblocks are identical and the sum of the absolute differences is zero. If there is a small amount of non-compensated motion, then the sum of the absolute differences is small, and if there is a large amount of non-compensated motion, then the sum of the absolute differences is large. The sum of weighted differences is an approximation to the sum of absolute differences, and in one embodiment, the sum of weighted differences is a faster and more efficient method of calculation than the sum of absolute differences;” paragraph 0023, “σi is a dimensionless measure of the current macroblock activity for one embodiment;” paragraph 0024, “the region in which the hand is moving has lots of activity due to non-compensated motion and to ensure that the spatial quality of the video is maintained, more bits are required by the moving hand than by a stationary hand.”).
Schmit is directed toward, “Embodiments include a codec for use in a videoconferencing or similar system includes a video encoder pipeline that has a pre-processor component that is optimized to detect faces and compress the facial video data in an optimum manner (abstract).” Martins is directed toward, “A source model in combination with an interest structure is provided to generate a quantization value for use in encoding a video signal. The interest structure is generated from a region of interest manually identified by a user viewing the video on an interactive user display or automatically by a system which recognizes the regions of interest automatically (abstract).” As can be easily seen by one of ordinary skill in the art before the effective filing date of the claimed invention, Schmit and Martins are directed toward similar methods of endeavor of video data analysis and region determination. Further, one of ordinary skill in the art before the effective filing date of the claimed invention would easily understand detecting motion across video data can be performed by comparing frame data where differences represent movement. Further, movement often is considered an area of interest in video being that there is some sort of changing feature when motion is present. Thus, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to ensure the moving regions can be analyzed, and feature can be preserved in bit rate determination.
Allowable Subject Matter
Claims 2-3 and 7 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claim 2:
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach methods of bit rate control and allocation. However, none of them alone or in any combination teaches calculating the person blend by calculating a person flatness based on texture confidence and a person reinforcement parameter. The closest prior art being Schmit discloses at column 5, line 59, “ The bitrate algorithm essentially uses the activity map data provided by the pre-processor to determine the activity of the face region relative to the average activity and allocate bits in accordance with what is important. In one embodiment, the bitrate parameter 606 is provided by the output of combinatorial circuit 212 to control the quantization and the fine grained quality of the face object detected by face detector 202. This allows enhanced detail to be provided to the face image.”
However, Schmit fails to disclose calculating the person blend by calculating a person flatness based on texture confidence and a person reinforcement parameter.
Claim 3:
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach methods of bit rate control and allocation. However, none of them alone or in any combination teaches calculating the motion blend by calculating a motion flatness based on texture confidence and a motion reinforcement parameter.
The closest prior art Schmit discloses at column 5, line 59, “ The bitrate algorithm essentially uses the activity map data provided by the pre-processor to determine the activity of the face region relative to the average activity and allocate bits in accordance with what is important. In one embodiment, the bitrate parameter 606 is provided by the output of combinatorial circuit 212 to control the quantization and the fine grained quality of the face object detected by face detector 202. This allows enhanced detail to be provided to the face image.”
However, Schmit fails to disclose calculating the motion blend by calculating a motion flatness based on texture confidence and a motion reinforcement parameter.
Claim 7:
The following is a statement of reasons for the indication of allowable subject matter: the closest prior arts of record teach methods of bit rate control and allocation.
However, none of them alone or in any combination teaches obtaining two gradient graphs through convolution, calculating at least two gradient directions, performing standard deviation on the gradient direction to obtain anisotropy levels, and further obtaining gradient magnitude and level of isotropy, and calculating a texture confidence based on the determined values.
The closest prior art Schmit discloses at column 5, line 59, “The bitrate algorithm essentially uses the activity map data provided by the pre-processor to determine the activity of the face region relative to the average activity and allocate bits in accordance with what is important,” where the activity map is read as the texture confidence
However, Schmit fails to disclose obtaining two gradient graphs through convolution, calculating at least two gradient directions, performing standard deviation on the gradient direction to obtain anisotropy levels, and further obtaining gradient magnitude and level of isotropy, and calculating a texture confidence based on the determined values.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
U.S. Patent No. 6,088,392 to Rosenberg discloses, “The system uses a prequantization process to reduce the computational load imposed by the quantizer optimization calculations required for feed-forward semantic coding bit-rate control (abstract).”
U.S. Publication No. 2006/0171457 to DeGarrido discloses, “A video compression algorithm for converting a digital video stream having video content into a compressed video bitstream, includes rate control to control the size of the compressed video bitstream (abstract).”
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Courtney J. Windsor whose telephone number is (571)272-3956. The examiner can normally be reached Monday - Friday 8:00 - 4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, John Villecco can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COURTNEY JOAN NELSON/Primary Examiner, Art Unit 2661
Prosecution Insights