DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-20 are pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Objections
Claim 2 is objected to because of the following informalities: In Pg. 1, line 19, “at least one frame of processed target image” should read –the at least one frame of the processed target image–. Appropriate correction is required.
Claim 3 is objected to because of the following informalities: In Pg. 2, line 4, “the salient area” should read –the at least one salient area–. Appropriate correction is required.
Claim 4 is objected to because of the following informalities:
In Pg. 2, lines 6-7, “processed target image” should read –the processed target image–.
In Pg. 2, line 9, “fused image” should read –a fused image–.
Appropriate correction is required.
Claim 5 is objected to because of the following informalities:
In Pg. 2, line 19, “an enhance image and a wakened image” should read –an enhanced image and a weakened image–.
In Pg. 3, lines 1-2, “at least frame of the processed target image” should read –the at least one frame of the processed target image–.
Appropriate correction is required.
Claim 11 is objected to because of the following informalities: In Pg. 5, line 1, “at least one frame of processed target image” should read –the at least one frame of the processed target image–. Appropriate correction is required.
Claim 12 is objected to because of the following informalities: In Pg. 5, line 9, “the salient area” should read –the at least one salient area–. Appropriate correction is required.
Claim 13 is objected to because of the following informalities:
In Pg. 5, line 15, “an enhance image and a wakened image” should read –an enhanced image and a weakened image–.
In Pg. 5, lines 22-23, “at least frame of the processed target image” should read –the at least one frame of the processed target image–.
Appropriate correction is required.
Claim 19 is objected to because of the following informalities: In Pg. 7, line 22, “at least one frame of processed target image” should read –the at least one frame of the processed target image–. Appropriate correction is required.
Claim 20 is objected to because of the following informalities: In Pg. 8, line 6, “the salient area” should read –the at least one salient area–. Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “acquisition module”, “detection module”, “differential processing module”, and “encoding module” in claim 10, “differential processing module” in claims 11-13, “detection module” and “visual acquisition device” in claim 16, and “encoding module” in claim 17.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 11-17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 11 recites the limitation "the device" in Pg. 4, line 20. It is unclear and indefinite which device is being referred to here (i.e., data processing device or electronic device).
Claim 12 recites the limitation "the device" in Pg. 5, line 4. It is unclear and indefinite which device is being referred to here (i.e., data processing device or electronic device).
Claim 13 recites the limitation "the device" in Pg. 5, line 11. It is unclear and indefinite which device is being referred to here (i.e., data processing device or electronic device).
Claim 14 recites the limitation "the device" in Pg. 6, line 1. It is unclear and indefinite which device is being referred to here (i.e., data processing device or electronic device).
Claim 15 recites the limitation "the device" in Pg. 6, line 7. It is unclear and indefinite which device is being referred to here (i.e., data processing device or electronic device).
Claim 16 recites the limitation "the device" in Pg. 6, line 11. It is unclear and indefinite which device is being referred to here (i.e., data processing device or electronic device).
Claim 17 recites the limitation "the device" in Pg. 6, line 18. It is unclear and indefinite which device is being referred to here (i.e., data processing device or electronic device).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 18-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claims do not fall within at least of the four categories of patent eligible subject matter because the broadest reasonable interpretation of the “computer readable storage medium” of claim 18 encompasses signals per se. The specification states that a “computer-readable storage medium 402 may be configured to store instructions” in Para. 00113, and “computer-readable storage medium can include a memory…or various apparatuses including one or any combination thereof” in Para. 00115 which clearly includes propagating electromagnetic waves. The further recitation of “instructions” in claim 18 only serves to limit the content carried by the electromagnetic waves. As understood in light of the specification, the broadest reasonable interpretation of claim 18 encompasses signals which are not within one of the four statutory categories of invention. See MPEP2106.03 I. It is suggested that claim 18 be amended to recite a “non-transitory” computer readable storage medium to overcome this rejection. Claims 19-20 depend on claim 18 and are also rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 7, 10-11, 15, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Dumitras et al. (US 2005/0036704 A1, hereinafter “Dumitras”).
Regarding claim 1, Dumitras teaches, a data processing method comprising (Para. 0006: “a foreground/background differentiation pre-processing method that performs filtering differently on a foreground region of a video frame in a video sequence than on a background region of the video frame”):
obtaining at least one frame of an original image (As shown in Figs. 1 and 2, an original video sequence is input for processing (i.e., pre-processing); Para. 0006: a region (i.e., foreground region) of a video frame in a video sequence is pre-processed; Fig. 10: in step 1005, an original video sequence is received, and in step 1010, frames are set);
performing visual saliency detection on each frame of the at least one frame of the original image to obtain visual saliency data of each frame of the original image (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0165-192; Note: the Examiner interprets the binary mask for the foreground region and the background region, in Para. 0165 for example, in which the foreground region is the important region and the background region is unimportant region, as visual saliency detection/data);
performing differential processing on different positions of each frame of the original image to obtain at least one frame of a processed target image based on the visual saliency data (Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering, for example, as differential processing);
and encoding the processed target image (Para. 0045: “The encoding component 110 then receives the pre-processed video sequence and encodes (i.e., compresses) the pre-processed video sequence to produce a pre-processed and compressed video sequence”; Para. 0178-0192).
Dumitras discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine the embodiments for obtaining an image frame, performing visual saliency detection, performing differential processing, and encoding the image since different embodiments may be used in any combination with any other embodiment of the present invention (Dumitras, Para. 0008) and in order to allow for simple and fast filtering in real-time applications and bit rate reduction of the compressed video sequence (Dumitras, Para. 0007). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 1.
Regarding claim 2, Dumitras teaches the limitations as explained above in claim 1.
Dumitras further teaches, the method of claim 1 (see claim 1 above), wherein performing the visual saliency detection on each frame of the original image in the at least one frame of the original image to obtain the visual saliency data of each frame of the original image based on the visual saliency data includes (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0165-192; Note: the Examiner interprets the binary mask for the foreground region and the background region, in Para. 0165 for example, in which the foreground region is the important region and the background region is unimportant region, as visual saliency detection/data):
dividing each frame of the original image into areas to obtain a plurality of image areas based on the visual saliency data (Para. 0006: “The method includes identifying pixel locations in the video frame having pixel values that match characteristics of human skin. A bounding shape is then determined for each contiguous grouping of matching pixel locations (i.e., regions-of-interest), the bounding shape enclosing all or a portion of the contiguous grouping of matching pixel locations. The totality of all pixel locations of the video frame contained in a bounding shape is referred to as a foreground region. Any pixel locations in the video frame not contained within the foreground region comprises a background region”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region…”; Note: the Examiner interprets the bounding shape for each region of interest as an example of dividing the image frame);
performing differential image quality adjustment processing on different image areas in the plurality of image areas to obtain a plurality of processed image areas (Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering of the foreground and background regions as an example of performing differential image quality adjustment);
and obtaining at least one frame of processed target image based on the plurality of processed image areas (Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0177; Note: the Examiner interprets, for example, the single filtered frame containing the foreground and background regions as the at least one frame of the processed target image).
Regarding claim 7, Dumitras teaches the limitations as explained above in claim 1.
Dumitras further teaches, the method of claim 1 (see claim 1 above), wherein: a difference in image bit rates before and after the differential processing of each frame of the original image is less than a set threshold (Para. 0034: “after compression of the pre-processed video sequence, the bit rate of the pre-processed and compressed video sequence may be lower than the bit rate of the video sequence after compression but without pre-processing; Para. 0041; Para. 0045: “As such, the bit rate of the pre-processed and compressed video sequence is lower than the bit rate that would be obtained by compressing the original video sequence (without pre-preprocessing) with an identical compression method using the encoding component 110. The bit rate of a video sequence reflects an amount of binary coded data required to represent the video sequence over a given period of time and is typically measured in kilobits per second”; Note: the Examiner interprets the bit rate of the pre-processed video sequence as the image bit rate before differential processing, the bit rate of the compressed video sequence as the image bit rate after differential processing, and the bit rate being lower than the bit rate (i.e., amount of binary coded data) of the video sequence without any pre-processing as less than a set threshold).
Regarding claim 10, Dumitras teaches, a data processing device, applied to an electronic device, comprising (Para. 0047: “FIG. 2 illustrates a block diagram of video pre-processing component 105 with separate temporal pre-filtering and spatial pre-filtering components 205 and 210, respectively. The video pre-processing component 105 receives an original video sequence comprised of multiple video frames and produces a pre-processed video sequence. In some embodiments, the temporal pre-filtering component 205 performs pre-processing operations on the received video sequence and sends the video sequence to the spatial pre-filtering component 210 for further pre-processing. In other embodiments, the spatial pre-filtering component 210 performs pre-processing operations on the received video sequence and sends the video sequence to the temporal pre-filtering component 205 for further pre-processing. In further embodiments, pre-processing is performed only by the temporal pre-filtering component 205 or only by the spatial pre-filtering component 210. In some embodiments, the temporal pre-filtering component 205 and the spatial pre-filtering component 210 are configured to perform particular functions through instructions of a computer program product having a computer readable medium”; Note: the Examiner interprets computer as an electronic processing device):
an acquisition module, the acquisition module being configured to obtain at least one frame of an original image (Para. 0047: “FIG. 2 illustrates a block diagram of video pre-processing component 105 with separate temporal pre-filtering and spatial pre-filtering components 205 and 210, respectively. The video pre-processing component 105 receives an original video sequence comprised of multiple video frames and produces a pre-processed video sequence”; As shown in Figs. 1 and 2, an original video sequence is input for processing (i.e., pre-processing), and there is a pre-processor 105; Para. 0006: a region (i.e., foreground region) of a video frame in a video sequence is pre-processed; Fig. 10: in step 1005, an original video sequence is received, and in step 1010, frames are set);
a detection module, the detection module being configured to perform visual saliency detection on each frame of the at least one frame of the original image to obtain visual saliency data of each frame of the original image (Para. 0166: “The foreground/background differentiation method 1000 may be performed, for example, by the spatial pre-filtering component 210 or the encoder component 110”; Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0165-192; Note: the Examiner interprets the binary mask for the foreground region and the background region, in Para. 0165 for example, in which the foreground region is the important region and the background region is unimportant region, as visual saliency detection/data);
a differential processing module, the differential processing module being configured to perform differential processing on different positions of each frame of the original image to obtain at least one frame of a processed target image based on the visual saliency data (Fig. 2: temporal pre-filtering component 205 and spatial pre-filtering component 210; Para. 0047; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering, for example, as differential processing);
and an encoding module, the encoding module being configured to encode the processed target image (Fig. 1: encoder 110; Para. 0045: “The encoding component 110 then receives the pre-processed video sequence and encodes (i.e., compresses) the pre-processed video sequence to produce a pre-processed and compressed video sequence”; Para. 0178-0192).
Dumitras discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine the embodiments for obtaining an image frame, performing visual saliency detection, performing differential processing, and encoding the image since different embodiments may be used in any combination with any other embodiment of the present invention (Dumitras, Para. 0008) and in order to allow for simple and fast filtering in real-time applications and bit rate reduction of the compressed video sequence (Dumitras, Para. 0007). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 10.
Regarding claim 11, Dumitras teaches the limitations as explained above in claim 10.
Dumitras further teaches, the device of claim 10 (see claim 10 above), wherein the differential processing module is further configured to (Fig. 2: temporal pre-filtering component 205 and spatial pre-filtering component 210; Para. 0047; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering, for example, as differential processing):
divide each frame of the original image into areas to obtain a plurality of image areas based on the visual saliency data (Para. 0006: “The method includes identifying pixel locations in the video frame having pixel values that match characteristics of human skin. A bounding shape is then determined for each contiguous grouping of matching pixel locations (i.e., regions-of-interest), the bounding shape enclosing all or a portion of the contiguous grouping of matching pixel locations. The totality of all pixel locations of the video frame contained in a bounding shape is referred to as a foreground region. Any pixel locations in the video frame not contained within the foreground region comprises a background region”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region…”; Note: the Examiner interprets the bounding shape for each region of interest as an example of dividing the image frame);
perform differential image quality adjustment processing on different image areas in the plurality of image areas to obtain a plurality of processed image areas (Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering of the foreground and background regions as an example of performing differential image quality adjustment);
and obtain at least one frame of processed target image based on the plurality of processed image areas (Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0177; Note: the Examiner interprets, for example, the single filtered frame containing the foreground and background regions as the at least one frame of the processed target image).
Regarding claim 15, Dumitras teaches the limitations as explained above in claim 10.
Dumitras further teaches, the device of claim 10 (see claim 10 above), wherein: a difference in image bit rates before and after the differential processing of each frame of the original image is less than a set threshold (Para. 0034: “after compression of the pre-processed video sequence, the bit rate of the pre-processed and compressed video sequence may be lower than the bit rate of the video sequence after compression but without pre-processing; Para. 0041; Para. 0045: “As such, the bit rate of the pre-processed and compressed video sequence is lower than the bit rate that would be obtained by compressing the original video sequence (without pre-preprocessing) with an identical compression method using the encoding component 110. The bit rate of a video sequence reflects an amount of binary coded data required to represent the video sequence over a given period of time and is typically measured in kilobits per second”; Note: the Examiner interprets the bit rate of the pre-processed video sequence as the image bit rate before differential processing, the bit rate of the compressed video sequence as the image bit rate after differential processing, and the bit rate being lower than the bit rate (i.e., amount of binary coded data) of the video sequence without any pre-processing as less than a set threshold).
Regarding claim 18, Dumitras teaches, a computer readable storage medium, storing computer instructions, when executed by one or more processors, the computer instructions perform a data processing method comprising (Para. 0047: “FIG. 2 illustrates a block diagram of video pre-processing component 105 with separate temporal pre-filtering and spatial pre-filtering components 205 and 210, respectively…In some embodiments, the temporal pre-filtering component 205 and the spatial pre-filtering component 210 are configured to perform particular functions through instructions of a computer program product having a computer readable medium”; claim 22: “A computer program product having a computer readable medium having computer program instructions recorded thereon, the computer program product comprising”; Note: the Examiner interprets the pre-processing component and the filtering components as processors):
obtaining at least one frame of an original image (As shown in Figs. 1 and 2, an original video sequence is input for processing (i.e., pre-processing); Para. 0006: a region (i.e., foreground region) of a video frame in a video sequence is pre-processed; Fig. 10: in step 1005, an original video sequence is received, and in step 1010, frames are set);
performing visual saliency detection on each frame of the at least one frame of the original image to obtain visual saliency data of each frame of the original image (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0165-192; Note: the Examiner interprets the binary mask for the foreground region and the background region, in Para. 0165 for example, in which the foreground region is the important region and the background region is unimportant region, as visual saliency detection/data);
performing differential processing on different positions of each frame of the original image to obtain at least one frame of a processed target image based on the visual saliency data (Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering, for example, as differential processing);
and encoding the processed target image (Para. 0045: “The encoding component 110 then receives the pre-processed video sequence and encodes (i.e., compresses) the pre-processed video sequence to produce a pre-processed and compressed video sequence”; Para. 0178-0192).
Dumitras discloses and teaches the above limitations in different embodiments. It would have been obvious, before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine the embodiments for obtaining an image frame, performing visual saliency detection, performing differential processing, and encoding the image since different embodiments may be used in any combination with any other embodiment of the present invention (Dumitras, Para. 0008) and in order to allow for simple and fast filtering in real-time applications and bit rate reduction of the compressed video sequence (Dumitras, Para. 0007). Therefore, one of ordinary skill in the art would be capable to have combined the embodiments as claimed by known methods, and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned reasons that the Examiner has reached a conclusion of obviousness with respect to claim 18.
Regarding claim 19, Dumitras teaches the limitations as explained above in claim 18.
Dumitras further teaches, the computer readable storage medium of claim 18 (see claim 18 above), wherein performing the visual saliency detection on each frame of the original image in the at least one frame of the original image to obtain the visual saliency data of each frame of the original image based on the visual saliency data includes (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0165-192; Note: the Examiner interprets the binary mask for the foreground region and the background region, in Para. 0165 for example, in which the foreground region is the important region and the background region is unimportant region, as visual saliency detection/data):
dividing each frame of the original image into areas to obtain a plurality of image areas based on the visual saliency data (Para. 0006: “The method includes identifying pixel locations in the video frame having pixel values that match characteristics of human skin. A bounding shape is then determined for each contiguous grouping of matching pixel locations (i.e., regions-of-interest), the bounding shape enclosing all or a portion of the contiguous grouping of matching pixel locations. The totality of all pixel locations of the video frame contained in a bounding shape is referred to as a foreground region. Any pixel locations in the video frame not contained within the foreground region comprises a background region”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region…”; Note: the Examiner interprets the bounding shape for each region of interest as an example of dividing the image frame);
performing differential image quality adjustment processing on different image areas in the plurality of image areas to obtain a plurality of processed image areas (Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering of the foreground and background regions as an example of performing differential image quality adjustment);
and obtaining at least one frame of processed target image based on the plurality of processed image areas (Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0177; Note: the Examiner interprets, for example, the single filtered frame containing the foreground and background regions as the at least one frame of the processed target image).
Claims 3, 12, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Dumitras et al. (US 2005/0036704 A1, hereinafter “Dumitras”) in view of Hu (US 2021/0366438 A1).
Regarding claim 3, Dumitras teaches the limitations as explained above in claim 2.
Dumitras further teaches, the method of claim 2 (see claim 2 above), wherein: the plurality of image areas includes at least one salient area, and performing the differential image quality adjustment processing on different image areas in the plurality of image areas includes (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets the foreground region (i.e., important region) as the salient area, and filtering of the foreground and background regions as an example of performing differential image quality adjustment):
performing image quality enhancement processing on the at least one salient area (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps… 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering of the foreground region as an example of performing image quality enhancement on a salient area),
Dumitras does not expressly disclose the following limitation: and image quality weakening processing or no processing on other image areas, the other image areas being each image area in the plurality of image areas other than the salient area.
However, Hu teaches, and image quality weakening processing or no processing on other image areas, the other image areas being each image area in the plurality of image areas other than the salient area (Para. 0032; Para. 0037: “Of course, the live broadcasting user may also perform image displaying weakening processing on the selected object in the projected image. For example, when the live broadcasting user selects to weaken a background, a background display region may be recognized from the layer data, and weakening adjustment may be performed on a display parameter of the background region to weaken displaying of the background region in a live broadcast image during live broadcasting”; Note: the Examiner select the weakening processing limitation and interprets the background as a region other than the salient region).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine performing image quality weakening on image areas other than the salient area as taught by Hu with the data processing method of Dumitras in order to not display a certain region for live broadcasting (Hu, Para. 0032). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 3.
Regarding claim 12, Dumitras teaches the limitations as explained above in claim 11.
Dumitras further teaches, the device of claim 11 (see claim 11 above), wherein: the plurality of image areas includes at least one salient area, and the differential processing module is further configured to (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets the foreground region (i.e., important region) as the salient area, and filtering of the foreground and background regions as an example of performing differential image quality adjustment):
perform image quality enhancement processing on the at least one salient area (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps… 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering of the foreground region as an example of performing image quality enhancement on a salient area),
Dumitras does not expressly disclose the following limitation: and image quality weakening processing or no processing on other image areas, the other image areas being each image area in the plurality of image areas other than the salient area.
However, Hu teaches, and image quality weakening processing or no processing on other image areas, the other image areas being each image area in the plurality of image areas other than the salient area (Para. 0032; Para. 0037: “Of course, the live broadcasting user may also perform image displaying weakening processing on the selected object in the projected image. For example, when the live broadcasting user selects to weaken a background, a background display region may be recognized from the layer data, and weakening adjustment may be performed on a display parameter of the background region to weaken displaying of the background region in a live broadcast image during live broadcasting”; Note: the Examiner select the weakening processing limitation and interprets the background as a region other than the salient region).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine performing image quality weakening on image areas other than the salient area as taught by Hu with the data processing method of Dumitras in order to not display a certain region for live broadcasting (Hu, Para. 0032). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 12.
Regarding claim 20, Dumitras teaches the limitations as explained above in claim 19.
Dumitras further teaches, the computer readable storage medium of claim 19 (see claim 19 above), wherein: the plurality of image areas includes at least one salient area, and performing the differential image quality adjustment processing on different image areas in the plurality of image areas includes (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets the foreground region (i.e., important region) as the salient area, and filtering of the foreground and background regions as an example of performing differential image quality adjustment):
performing image quality enhancement processing on the at least one salient area (Para. 0164: “Performing different filtering on different regions of the video frame allows a system to provide greater data reduction in unimportant background regions of the video frame while preserving sharp edges in regions-of-interest in the foreground region”; Para. 0029: “good video pre-processing system may also improve the visual quality of the decoded sequences…”; Para. 0031: “Spatial filtering is a pre-processing step used for anti-aliasing and smoothing (by removing details of a video frame that are unimportant for the perceived visual quality) and segmentation”; Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps… 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0189; Note: the Examiner interprets filtering of the foreground region as an example of performing image quality enhancement on a salient area),
Dumitras does not expressly disclose the following limitation: and image quality weakening processing or no processing on other image areas, the other image areas being each image area in the plurality of image areas other than the salient area.
However, Hu teaches, and image quality weakening processing or no processing on other image areas, the other image areas being each image area in the plurality of image areas other than the salient area (Para. 0032; Para. 0037: “Of course, the live broadcasting user may also perform image displaying weakening processing on the selected object in the projected image. For example, when the live broadcasting user selects to weaken a background, a background display region may be recognized from the layer data, and weakening adjustment may be performed on a display parameter of the background region to weaken displaying of the background region in a live broadcast image during live broadcasting”; Note: the Examiner select the weakening processing limitation and interprets the background as a region other than the salient region).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine performing image quality weakening on image areas other than the salient area as taught by Hu with the data processing method of Dumitras in order to not display a certain region for live broadcasting (Hu, Para. 0032). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 20.
Claims 4 is rejected under 35 U.S.C. 103 as being unpatentable over Dumitras et al. (US 2005/0036704 A1, hereinafter “Dumitras”) in view of “Learning Edge-Preserved Image Stitching from Large-Baseline Deep Homography” by Nie et al. (hereinafter “Nie”).
Regarding claim 4, Dumitras teaches the limitations as explained above in claim 2.
Dumitras further teaches, the method of claim 2 (see claim 2 above), wherein obtaining at least one frame of processed target image based on the plurality of processed image areas includes (Para. 0165: “The foreground/background differentiation method of the present invention includes five general steps: 1) identifying pixel locations in a video frame having pixel values that match color characteristics of human skin and identification of contiguous groupings of matching pixel locations (i.e., regions-of-interest); 2) determining a bounding shape for each region-of-interest, the totality of all pixel locations contained in a bounding shape comprising a foreground region and all other pixel locations in the frame comprising a background region; 3) creating a binary mask Mfg for the foreground region and a binary mask Mbg for the background region; 4) filtering the foreground and background regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0177; Note: the Examiner interprets, for example, the single filtered frame containing the foreground and background regions as the at least one frame of the processed target image):
performing image fusion processing on the plurality of processed image areas to obtain at least one frame of fused image (Para. 0165: “4) filtering the foreground and back ground regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0177),
and determining each frame of the fused image as the processed target image (Para. 0165: “4) filtering the foreground and back ground regions using different filtering methods or parameters using the binary masks; and 5) combining the filtered foreground and background regions into a single filtered frame”; Paras. 0176-0177; Para. 0045: The encoding component 110 then receives the pre-processed video sequence and encodes (i.e., compresses) the pre-processed video sequence to produce a pre-processed and compressed Video Sequence. Pre-filtering methods performed by the pre-processing component 105 allows removal of noise and details from the original video sequence thus allowing for greater compression of the pre-processed video sequence by the encoding component 110”).
Dumitras does not expressly disclose the following limitation: an edge continuity between each image area in the fused image meeting a set requirement.
However, Nie teaches, an edge continuity between each image area in the fused image meeting a set requirement (Pg. 2: “Our edge-preserveddeformation module overcomes this problem bylearning to correct the discontinuity around the edges (Fig.1 (e)), contributing to a visually pleasing and edge-continuitystitched result”; Pg. 10: “(2) After ablating this branch, the edges of the stitched images is not discontinuous as shown in Fig. 12 (a). With this branch (Fig. 12 (b)), the network further learns to smooth the discontinuous edges, contributing to visually pleasing and edge-continuity stitched results”; Note: the Examiner interprets “visually pleasing and edge-continuity stitched results” as the fused image meeting a set requirement).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine there being an edge continuity between image areas of a fused image as taught by Nie with the data processing method of Dumitras in order to eliminate ghosting effects (Nie, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 4.
Claims 8 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Dumitras et al. (US 2005/0036704 A1, hereinafter “Dumitras”) in view of “SalFBNet: Learning pseudo-saliency distribution via feedback convolutional networks” by Ding et al. (hereinafter “Ding”).
Regarding claim 8, Dumitras teaches the limitations as explained above in claim 1.
Dumitras does not expressly disclose the following limitations: wherein performing the visual saliency detection on each frame of the original image to obtain the visual saliency data of each frame of the original image includes: inputting each frame of the original image into a pre-trained visual saliency model for the visual saliency detection to obtain the visual saliency data of each frame of the original image, the visual saliency model being trained based on a saliency data set collected by a visual acquisition device and a pseudo-saliency data set.
However, Ding teaches, wherein performing the visual saliency detection on each frame of the original image to obtain the visual saliency data of each frame of the original image includes: inputting each frame of the original image into a pre-trained visual saliency model for the visual saliency detection to obtain the visual saliency data of each frame of the original image, the visual saliency model being trained based on a saliency data set collected by a visual acquisition device and a pseudo-saliency data set (Abstract: “In this work, we propose a feedback-recursive convolutional framework (SalFBNet) for saliency detection. The proposed feedback model can learn abundant contextual representations by bridging a recursive pathway from higher-level feature blocks to low-level layers. Moreover, we create a large-scale Pseudo-Saliency dataset to alleviate the problem of data deficiency in saliency detection. We first use the proposed feedback model to learn saliency distribution from pseudo-ground-truth. Afterwards, we fine-tune the feedback model on existing eye-fixation datasets”; Pg. 1-2; Pg. 6: “For pseudo-saliency annotation, we first select 150,000 color images from widely-used ImageNet [28] dataset and 26,880 color images from SOD datasets. In our experiment, SOD datasets include CSSD [41], ECSSD [42], HKU-IS [43], MSRA-B [44], MSRA10K [45], and THUR15K [46]. Afterwards, we choose M=5 pre-trained saliency models to annotate these images. The pre-trained models include DeepGazeIIE [19], UNISAL [16], MSINet [23], EMLNet[24], CASNetII [25].We directl yuse their pre-trained weights and default settings for inference of saliency distribution. Therefore, we create a large-scale Pseudo-Saliency dataset containing 176,880 color images and corresponding pseudo-ground truths”; Fig. 1:
PNG
media_image1.png
441
711
media_image1.png
Greyscale
).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine inputting frames/images into a pre-trained visual saliency model for visual saliency detection to obtain visual saliency data of each frame in which the visual saliency model is trained on a saliency data set collected by a visual acquisition device and a pseudo-saliency data set as taught by Ding with the data processing method of Dumitras in order to learn abundant contextual features for saliency prediction (Ding, Pg. 11, 5. Conclusion). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 8.
Regarding claim 16, Duumitras teaches the limitationas as explained above in claim 10.
Dumitras does not expressly disclose the following limitations: wherein the detection module is further configured to: input each frame of the original image into a pre-trained visual saliency model for the visual saliency detection to obtain the visual saliency data of each frame of the original image, the visual saliency model being trained based on a saliency data set collected by a visual acquisition device and a pseudo-saliency data set.
However, Ding teaches, wherein the detection module is further configured to: input each frame of the original image into a pre-trained visual saliency model for the visual saliency detection to obtain the visual saliency data of each frame of the original image, the visual saliency model being trained based on a saliency data set collected by a visual acquisition device and a pseudo-saliency data set (Abstract: “In this work, we propose a feedback-recursive convolutional framework (SalFBNet) for saliency detection. The proposed feedback model can learn abundant contextual representations by bridging a recursive pathway from higher-level feature blocks to low-level layers. Moreover, we create a large-scale Pseudo-Saliency dataset to alleviate the problem of data deficiency in saliency detection. We first use the proposed feedback model to learn saliency distribution from pseudo-ground-truth. Afterwards, we fine-tune the feedback model on existing eye-fixation datasets”; Pg. 1-2; Pg. 6: “For pseudo-saliency annotation, we first select 150,000 color images from widely-used ImageNet [28] dataset and 26,880 color images from SOD datasets. In our experiment, SOD datasets include CSSD [41], ECSSD [42], HKU-IS [43], MSRA-B [44], MSRA10K [45], and THUR15K [46]. Afterwards, we choose M=5 pre-trained saliency models to annotate these images. The pre-trained models include DeepGazeIIE [19], UNISAL [16], MSINet [23], EMLNet[24], CASNetII [25].We directly use their pre-trained weights and default settings for inference of saliency distribution. Therefore, we create a large-scale Pseudo-Saliency dataset containing 176,880 color images and corresponding pseudo-ground truths”; Fig. 1:
PNG
media_image1.png
441
711
media_image1.png
Greyscale
).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine inputting frames/images into a pre-trained visual saliency model for visual saliency detection to obtain visual saliency data of each frame in which the visual saliency model is trained on a saliency data set collected by a visual acquisition device and a pseudo-saliency data set as taught by Ding with the data processing method of Dumitras in order to learn abundant contextual features for saliency prediction (Ding, Pg. 11, 5. Conclusion). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 16.
Claims 9 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Dumitras et al. (US 2005/0036704 A1, hereinafter “Dumitras”) in view of Wan (CN 106162177 A, see provided machine translation).
Regarding claim 9, Dumitras teaches the limitations as explained above in claim 1.
Dumitras does not expressly disclose the following limitations: wherein encoding the processed target image includes: obtaining a first quantization parameter and a second quantization parameter of the target image, a first quantization parameter value being smaller than a second quantization parameter value; using the first quantization parameter for the encoding processing of the salient area in the target image; and using the second quantization parameter value for encoding processing of non-salient areas in the target image.
However, Wan teaches, wherein encoding the processed target image includes: obtaining a first quantization parameter and a second quantization parameter of the target image, a first quantization parameter value being smaller than a second quantization parameter value (Para. 0045: The quantization parameter of the region of interest and the region of non-interest is adjusted when encoding video frames. The region of interest is encoded using an encoding method where the quantization parameter is lower than that of the region of non-interest; Note: the Examiner interprets the quantization parameter for the region of interest as the first quantization parameter, and the quantization parameter for the region of non-interest as the second quantization parameter);
using the first quantization parameter for the encoding processing of the salient area in the target image (Para. 0039: the moving target is the foreground of the video frame, whereas the stationary or nearly stationary elements in the video frame are the background of the video frame; Para. 0040: the region where the moving target is located in the video frame is the region of interest; Para. 0045: The quantization parameter of the region of interest and the region of non-interest is adjusted when encoding video frames. The region of interest is encoded using an encoding method where the quantization parameter is lower than that of the region of non-interest; Note: the Examiner interprets the region of interest as the salient area, and the quantization parameter for the region of interest as the first quantization parameter);
and using the second quantization parameter value for encoding processing of non-salient areas in the target image (Para. 0039: the moving target is the foreground of the video frame, whereas the stationary or nearly stationary elements in the video frame are the background of the video frame; Para. 0040: the region where the moving target is located in the video frame is the region of interest; Para. 0045: The quantization parameter of the region of interest and the region of non-interest is adjusted when encoding video frames. The region of interest is encoded using an encoding method where the quantization parameter is lower than that of the region of non-interest; Note: the Examiner interprets the region of non-interest as the non-salient area, and the quantization parameter for the region of non-interest as the second quantization parameter).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine obtaining a first quantization parameter for encoding processing of the salient area, obtaining a second quantization parameter for encoding processing of non-salient areas, and the first quantization parameter being smaller than the second quantization parameter as taught by Wan with the data processing method of Dumitras in order to keep a relative high picture quality in the region in which the moving target is positioned and to lower the fidelity of the non-region of interest (Wan, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 9.
Regarding claim 17, Dumitras teaches the limitations as explained above in claim 10.
Dumitras does not expressly disclose the following limitations: wherein the encoding module is further configured to: obtain a first quantization parameter and a second quantization parameter of the target image, a first quantization parameter value being smaller than a second quantization parameter value; use the first quantization parameter for the encoding processing of the salient area in the target image; and use the second quantization parameter value for encoding processing of non-salient areas in the target image.
However, Wan teaches, wherein the encoding module is further configured to: obtain a first quantization parameter and a second quantization parameter of the target image, a first quantization parameter value being smaller than a second quantization parameter value (Para. 0015: “The encoding module is used to encode the video frame”; Fig. 8 and Para. 0100: encoding module 830; Para. 0045: The quantization parameter of the region of interest and the region of non-interest is adjusted when encoding video frames. The region of interest is encoded using an encoding method where the quantization parameter is lower than that of the region of non-interest; Note: the Examiner interprets the quantization parameter for the region of interest as the first quantization parameter, and the quantization parameter for the region of non-interest as the second quantization parameter);
use the first quantization parameter for the encoding processing of the salient area in the target image (Para. 0039: the moving target is the foreground of the video frame, whereas the stationary or nearly stationary elements in the video frame are the background of the video frame; Para. 0040: the region where the moving target is located in the video frame is the region of interest; Para. 0045: The quantization parameter of the region of interest and the region of non-interest is adjusted when encoding video frames. The region of interest is encoded using an encoding method where the quantization parameter is lower than that of the region of non-interest; Note: the Examiner interprets the region of interest as the salient area, and the quantization parameter for the region of interest as the first quantization parameter);
and use the second quantization parameter value for encoding processing of non-salient areas in the target image (Para. 0039: the moving target is the foreground of the video frame, whereas the stationary or nearly stationary elements in the video frame are the background of the video frame; Para. 0040: the region where the moving target is located in the video frame is the region of interest; Para. 0045: The quantization parameter of the region of interest and the region of non-interest is adjusted when encoding video frames. The region of interest is encoded using an encoding method where the quantization parameter is lower than that of the region of non-interest; Note: the Examiner interprets the region of non-interest as the non-salient area, and the quantization parameter for the region of non-interest as the second quantization parameter).
It would have been obvious before the effective filing date of the claimed invention, to one of ordinary skill in the art, to combine obtaining a first quantization parameter for encoding processing of the salient area, obtaining a second quantization parameter for encoding processing of non-salient areas, and the first quantization parameter being smaller than the second quantization parameter as taught by Wan with the data processing method of Dumitras in order to keep a relative high picture quality in the region in which the moving target is positioned and to lower the fidelity of the non-region of interest (Wan, Abstract). Therefore, one of ordinary skill in the art would be capable to have combined the elements as claimed by known methods and that in combination, each element merely performs the same function as it does separately. It is for at least the aforementioned that the Examiner has reached a conclusion of obviousness with respect to claim 17.
Allowable Subject Matter
Claims 5 and 6 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 13 and 14 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action and to include all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Yu (US 2022/0382053 A1)
Rijnders (US 2018/0240221 A1)
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniella M. DiGuglielmo whose telephone number is (571)272-0183. The examiner can normally be reached Monday - Friday 8:00 AM - 4:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daniella M. DiGuglielmo/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights