DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers submitted under 35 U.S.C. 119(a)-(d), which papers have been placed of record in the file.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 01/17/2025 and 03/03/2026 are being considered by the examiner.
Specification
The disclosure is objected to because of the following informalities:
The specification submitted on 01/17/2025 discloses in paragraph [25] “The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings.” However, no drawings have been filed with the application and are not located in the file wrapper. As such, the disclosure is objected to for failing to submit the disclosed drawings with the specification. The Office respectfully requests drawings be filed with the application in order to enhance the examination process.
Appropriate correction is required.
Claim Objections
Claims 8, 11, and 18 are objected to because of the following informalities:
In claim 8, line 5-6, the term “loss of the identified plurality of overlapped regions;” should be changed to “loss of the identified plurality of overlapping pixels;” in order to avoid an insufficient antecedent issue and prevent a rejection under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph issues.
In claim 11, line 8, the term “associated with the video,” should be changed to “associated with the video;” in order to avoid a sentence run on.
In claim 11, line 9, the term “into one or more regions,” should be changed to “into one or more regions;” in order to avoid a sentence run on.
In claim 11, line 11, the term “of the one or more regions,” should be changed to “of the one or more regions;” in order to avoid a sentence run on.
In claim 11, line 13, the term “first enhancement parameters,” should be changed to “first enhancement parameters;” in order to avoid a sentence run on.
In claim 11, line 15, the term “frames of the captured video, and” should be changed to “frames of the captured video; and” in order to avoid a sentence run on.
In claim 18, line 3, the term “one or more region masks,” should be changed to “one or more region masks;” in order to avoid a sentence run on.
In claim 18, line 5, the term “and the one or more region masks,” should be changed to “and the one or more region masks; and” in order to avoid a sentence run on.
In claim 18, line 7, the term “plurality of overlapped regions,” should be changed to “plurality of overlapped regions,;” in order to avoid a sentence run on.
In claim 18, line 10, the term “and the perceptual loss, and” should be changed to “and the perceptual loss; and” in order to avoid a sentence run on.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that use the word “means” or “step” but are nonetheless not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph because the claim limitation(s) recite(s) sufficient structure, materials, or acts to entirely perform the recited function.
Claims 4, 9, and 14 recites limitations that use words like “means” (or “step”) or similar terms with functional language but do not invoke 35 U.S.C. 112(f):
Claim 4; recites the limitation, “an exposure synthesis for……,” [Line 3].
Claim 4; recites the limitation, “motion blur parameters for……,” [Line 5].
Claim 9; recites the limitation, “user interface for……,” [Line 4].
Claim 14; recites the limitation, “an exposure synthesis for……,” [Line 3].
Claim 14; recites the limitation, “motion blur parameters for……,” [Line 5].
Such claim limitation(s) is/are:
“exposure synthesis….” has a structure associated with it a computer, since it is a computer vision/digital imaging technique to combine images.
“motion blur parameters….” has a structure associated with it a number/value.
“user interface….” has a structure associated with it a user interface.
Because this/these claim limitation(s) is/are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are not being interpreted to cover only the corresponding structure, material, or acts described in the specification as performing the claimed function, and equivalents thereof.
If applicant intends to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to remove the structure, materials, or acts that performs the claimed function; or (2) present a sufficient showing that the claim limitation(s) does/do not recite sufficient structure, materials, or acts to perform the claimed function.
Claims 1 and 11 recite limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
Claim 1; recites the limitation, “via a user equipment (UE)…..” [Line 2].
Claim 11; recites the limitation, “via a user equipment (UE)…..” [Line 6].
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claims 1 and 11;
(i) “user equipment” (Paragraph [43]- Figure 1 illustrates a block diagram of a user equipment (UE) 100 comprising a system 102 for enhancing a quality of a video, according to an embodiment of the disclosure. In an embodiment of the disclosure, the system 102 may be hosted on the UE 100. In an exemplary embodiment of the disclosure, the UE 100 may correspond to a smartphone, a laptop computer, a desktop computer, a wearable device, and the like. In an example embodiment, the system 102 may be hosted on a server. In this scenario, the UE 100 may access the system 102 hosted on the server to enhance the quality of the video. The system 102 may include one or more processors 104, a plurality of modules 106, a memory 108, and an input/output (1/O) interface 109. The user equipment thus has sufficient structure or material wherein is a computer.).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3 and 11-13 are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20120008832 A1), hereinafter referenced as WANG, in view of BORDER et al. (US 20100231738 A1), hereinafter referenced as BORDER.
Regarding claim 1, WANG explicitly teaches a method for capturing a video with enhanced quality (Fig. 1. Paragraph [0008]-WANG discloses the present invention provides a system and method for selective, local enhancement of images for object recognition.), the method comprising:
capturing a reference image (Fig. 1, called Video Input. Paragraph [0009]-WANG discloses video processor 10 may also include local memory for storing image input provided by an external device (e.g., video capture device, image capture device, data storage device, etc.) as well as for storing various values calculated as part of the analysis of the image. In addition, the image input may be original image data (i.e., unenhanced) or may be video data that has already been globally enhanced or otherwise converted from an original state to another state (wherein the video input data is a reference image).) via a user equipment (UE) (Fig. 1. Paragraph [0024]-WANG discloses the disclosed invention may therefore be embodied in the form of computer or controller implemented processes and apparatuses for practicing those processes. Further in paragraph [0024]-WANG discloses video processor 10 may be incorporated with a general purpose processor such as those employed on personal computers, or may be a stand-alone video processor responsible only for analyzing video data (wherein a computer is user equipment).),
wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video (Fig. 1. Paragraph [0009]-WANG discloses video processor 10 may also include local memory for storing image input provided by an external device (e.g., video capture device, image capture device, data storage device, etc.) as well as for storing various values calculated as part of the analysis of the image. In addition, the image input may be original image data (i.e., unenhanced) or may be video data that has already been globally enhanced or otherwise converted from an original state to another state.);
segmenting the captured reference image into one or more regions (Fig. 1. Paragraph [0013]-WANG discloses object detection module 12 provides as an output a region of interest (ROI) that defines a localized portion of the received global image (wherein a localized portion region of interest is a segmented region).);
receiving one or more first enhancement parameters for a first region of the one or more regions (Fig. 1. Paragraph [0018]-WANG discloses VQE module 16 may also selectively adjust parameters associated with the selected VQE function based on the estimated VQM value. In general, the parameters determine how much `enhancement` to apply to the ROI. For instance, in response to a VQM value that indicates a highly blurred image, the parameter associated with the sharpness enhancement function or super-resolution function would provide significant sharpness enhancement.);
WANG fails to explicitly teach initiating a capture of the video based on the one or more first enhancement parameters; identifying a plurality of pixels associated with the first region in each of the plurality of frames of the captured video; and applying the one or more first enhancement parameters to the identified plurality of pixels associated with the first region in each of the plurality of frames.
However, BORDER explicitly teaches initiating a capture of the video based on the one or more first enhancement parameters (Fig. 1, #125 called capture video image data with different frame rates in regions where different levels of motion are present. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshold (wherein the rate of capture is an enhancement parameter and a capture rate of the first region is the first enhancement parameter).);
identifying a plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the first region in each of the plurality of frames of the captured video (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame (wherein capturing the different regions is identifying the pixels with the different regions).); and
applying the one or more first enhancement parameters to the identified plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the first region (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame.) in each of the plurality of frames (Fig. 1. Paragraph [0026]-BORDER discloses the capture rate of the first region is selected to reduce the motion blur in each video frame.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of BORDER of initiating a capture of the video based on the one or more first enhancement parameters; identifying a plurality of pixels associated with the first region in each of the plurality of frames of the captured video; and applying the one or more first enhancement parameters to the identified plurality of pixels associated with the first region in each of the plurality of frames.
Wherein having WANG’s method of video quality enhancement having initiating a capture of the video based on the one or more first enhancement parameters; identifying a plurality of pixels associated with the first region in each of the plurality of frames of the captured video; and applying the one or more first enhancement parameters to the identified plurality of pixels associated with the first region in each of the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
Regarding claim 2, WANG in view of BORDER explicitly teach the method as claimed in claim 1, further comprising:
WANG fails to explicitly teach receiving one or more second enhancement parameters for a second region of the one or more regions; initiating the capture of the video based on the one or more second enhancement parameters; identifying a plurality of pixels associated with the second region in each of the plurality of frames of the captured video; and applying the one or more second enhancement parameters to the identified plurality of pixels associated with the second region in each of the plurality of frames.
However, BORDER explicitly teaches receiving one or more second enhancement parameters (Fig. 1. Paragraph [0026]-BORDER discloses in Step 115, capture rates for different regions within the video frames are assigned based on the determined relative speed of motion in the scene and the corresponding regions of the video frames. Regions in the video frames with slow or no motion, are assigned a relatively slower capture rate or maintained at the base capture rate, whereas, regions with rapid motion are assigned a relatively faster capture rate (wherein a slower or maintained rate of capture is a second enhancement parameter).) for a second region of the one or more regions (Fig. 1. Paragraph [0034]-BORDER discloses as rapid motion is identified in first regions of the video frames and the assigned capture rate is increased for the rapid motion regions, second regions of the video frames are identified which have slow or no motion present and the capture rate is decreased for the regions with slow or no motion.);
initiating the capture of the video (Fig. 1, #125 called capture video image data with different frame rates in regions where different levels of motion are present. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshold (wherein the rate of capture is an enhancement parameter and a capture rate of the second region is the second enhancement parameter).) based on the one or more second enhancement parameters (Fig. 1. Paragraph [0026]-BORDER discloses in Step 115, capture rates for different regions within the video frames are assigned based on the determined relative speed of motion in the scene and the corresponding regions of the video frames. Regions in the video frames with slow or no motion, are assigned a relatively slower capture rate or maintained at the base capture rate, whereas, regions with rapid motion are assigned a relatively faster capture rate (wherein a slower or maintained rate of capture is a second enhancement parameter).);
identifying a plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the second region in each of the plurality of frames of the captured video (Fig. 1, #125 called capture video image data with different frame rates in regions where different levels of motion are present. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame (wherein capturing the different regions is identifying the pixels with the different regions).); and
applying the one or more second enhancement parameters to the identified plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the second region in each of the plurality of frames (Fig. 1. Paragraph [0034]-BORDER discloses as rapid motion is identified in first regions of the video frames and the assigned capture rate is increased for the rapid motion regions, second regions of the video frames are identified which have slow or no motion present and the capture rate is decreased for the regions with slow or no motion (wherein a region is a group of pixels).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of BORDER of receiving one or more second enhancement parameters for a second region of the one or more regions; initiating the capture of the video based on the one or more second enhancement parameters; identifying a plurality of pixels associated with the second region in each of the plurality of frames of the captured video; and applying the one or more second enhancement parameters to the identified plurality of pixels associated with the second region in each of the plurality of frames.
Wherein having WANG’s method of video quality enhancement having receiving one or more second enhancement parameters for a second region of the one or more regions; initiating the capture of the video based on the one or more second enhancement parameters; identifying a plurality of pixels associated with the second region in each of the plurality of frames of the captured video; and applying the one or more second enhancement parameters to the identified plurality of pixels associated with the second region in each of the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
Regarding claim 3, WANG in view of BORDER explicitly teach the method as claimed in claim 1,
WANG fails to explicitly teach wherein a field of view (FOV) of the reference image includes the one or more regions included in the video.
However, BORDER explicitly teaches wherein a field of view (FOV) of the reference image includes the one or more regions included in the video (Fig. 6, illustrates a field of view with one or more regions in a view (wherein #120 and #110 are regions. Paragraph [0022]-BORDER discloses FIG. 6 shows an illustration of a video frame 100 of a person fencing that has rapid motion in the region of the image 120 that contains the person's arm and the fencing foil, and slower motion in the region of the image 110 that contains the person's lower body. In an embodiment of the present invention, the local capture rate is increased in the regions (120) of the video frames 100 when the speed of motion relative to the exposure time in the region is above a motion blur threshold.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of BORDER of wherein a field of view (FOV) of the reference image includes the one or more regions included in the video.
Wherein having WANG’s method of video quality enhancement having wherein a field of view (FOV) of the reference image includes the one or more regions included in the video.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
Regarding claim 11, WANG explicitly teaches a system for capturing a video with enhanced quality (Fig. 1. Paragraph [0008]-WANG discloses the present invention provides a system and method for selective, local enhancement of images for object recognition.), the system comprising:
memory storing instructions (Fig. 1. Paragraph [0024]-WANG discloses the present invention can also be embodied in the form of computer program code containing instructions embodied in computer readable medium, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by video processor 10, the processor becoming an apparatus for practicing the invention.); and
one or more processors (Fig. 1. Paragraph [0024]-WANG discloses video processor 10 may be incorporated with a general purpose processor such as those employed on personal computers, or may be a stand-alone video processor responsible only for analyzing video data.),
wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to (Fig. 1. Paragraph [0024]-WANG discloses the present invention can also be embodied in the form of computer program code containing instructions embodied in computer readable medium, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by video processor 10, the processor becoming an apparatus for practicing the invention.):
capture a reference image (Fig. 1, called Video Input. Paragraph [0009]-WANG discloses video processor 10 may also include local memory for storing image input provided by an external device (e.g., video capture device, image capture device, data storage device, etc.) as well as for storing various values calculated as part of the analysis of the image. In addition, the image input may be original image data (i.e., unenhanced) or may be video data that has already been globally enhanced or otherwise converted from an original state to another state (wherein the video input data is a reference image).) via a user equipment (UE) (Fig. 1. Paragraph [0024]-WANG discloses video processor 10 may be incorporated with a general purpose processor such as those employed on personal computers, or may be a stand-alone video processor responsible only for analyzing video data (wherein a processor is user equipment).),
wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video (Fig. 1. Paragraph [0009]-WANG discloses video processor 10 may also include local memory for storing image input provided by an external device (e.g., video capture device, image capture device, data storage device, etc.) as well as for storing various values calculated as part of the analysis of the image. In addition, the image input may be original image data (i.e., unenhanced) or may be video data that has already been globally enhanced or otherwise converted from an original state to another state.),
segment the captured reference image into one or more regions (Fig. 1. Paragraph [0013]-WANG discloses object detection module 12 provides as an output a region of interest (ROI) that defines a localized portion of the received global image (wherein a localized portion region of interest is a segmented region).),
receive one or more first enhancement parameters for a first region of the one or more regions (Fig. 1. Paragraph [0018]-WANG discloses VQE module 16 may also selectively adjust parameters associated with the selected VQE function based on the estimated VQM value. In general, the parameters determine how much `enhancement` to apply to the ROI. For instance, in response to a VQM value that indicates a highly blurred image, the parameter associated with the sharpness enhancement function or super-resolution function would provide significant sharpness enhancement.),
WANG fails to explicitly teach initiate a capture of the video based on the one or more first enhancement parameters, identify a plurality of pixels associated with the first region in each of the plurality of frames of the captured video, and apply the one or more first enhancement parameters to the identified plurality of pixels associated with the first region in each of the plurality of frames.
However, BORDER explicitly teaches initiate a capture of the video based on the one or more first enhancement parameters (Fig. 1, #125 called capture video image data with different frame rates in regions where different levels of motion are present. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshold (wherein the rate of capture is an enhancement parameter and a capture rate of the first region is the first enhancement parameter).),
identify a plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the first region in each of the plurality of frames of the captured video (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame (wherein capturing the different regions is identifying the pixels with the different regions).), and
apply the one or more first enhancement parameters to the identified plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the first region (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame.) in each of the plurality of frames (Fig. 1. Paragraph [0026]-BORDER discloses the capture rate of the first region is selected to reduce the motion blur in each video frame.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of BORDER of initiate a capture of the video based on the one or more first enhancement parameters, identify a plurality of pixels associated with the first region in each of the plurality of frames of the captured video, and apply the one or more first enhancement parameters to the identified plurality of pixels associated with the first region in each of the plurality of frames.
Wherein having WANG’s system of video quality enhancement having initiate a capture of the video based on the one or more first enhancement parameters, identify a plurality of pixels associated with the first region in each of the plurality of frames of the captured video, and apply the one or more first enhancement parameters to the identified plurality of pixels associated with the first region in each of the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
Regarding claim 12, WANG in view of BORDER explicitly teach the system as claimed in claim 11,
WANG further explicitly teaches wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to (Fig. 1. Paragraph [0024]-WANG discloses the present invention can also be embodied in the form of computer program code containing instructions embodied in computer readable medium, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by video processor 10, the processor becoming an apparatus for practicing the invention.):
WANG fails to explicitly teach receive one or more second enhancement parameters for a second region of the one or more regions; initiate the capture of the video based on the one or more second enhancement parameters; identify a plurality of pixels associated with the second region in each of the plurality of frames of the captured video; and apply the one or more second enhancement parameters to the identified plurality of pixels associated with the second region in each of the plurality of frames.
However, BORDER explicitly teaches receive one or more second enhancement parameters (Fig. 1. Paragraph [0026]-BORDER discloses in Step 115, capture rates for different regions within the video frames are assigned based on the determined relative speed of motion in the scene and the corresponding regions of the video frames. Regions in the video frames with slow or no motion, are assigned a relatively slower capture rate or maintained at the base capture rate, whereas, regions with rapid motion are assigned a relatively faster capture rate (wherein a slower or maintained rate of capture is a second enhancement parameter).) for a second region of the one or more regions (Fig. 1. Paragraph [0034]-BORDER discloses as rapid motion is identified in first regions of the video frames and the assigned capture rate is increased for the rapid motion regions, second regions of the video frames are identified which have slow or no motion present and the capture rate is decreased for the regions with slow or no motion.);
initiate the capture of the video (Fig. 1, #125 called capture video image data with different frame rates in regions where different levels of motion are present. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshold (wherein the rate of capture is an enhancement parameter and a capture rate of the second region is the second enhancement parameter).) based on the one or more second enhancement parameters (Fig. 1. Paragraph [0026]-BORDER discloses in Step 115, capture rates for different regions within the video frames are assigned based on the determined relative speed of motion in the scene and the corresponding regions of the video frames. Regions in the video frames with slow or no motion, are assigned a relatively slower capture rate or maintained at the base capture rate, whereas, regions with rapid motion are assigned a relatively faster capture rate (wherein a slower or maintained rate of capture is a second enhancement parameter).);
identify a plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the second region in each of the plurality of frames of the captured video (Fig. 1, #125 called capture video image data with different frame rates in regions where different levels of motion are present. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame (wherein capturing the different regions is identifying the pixels with the different regions).); and
apply the one or more second enhancement parameters to the identified plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the second region in each of the plurality of frames (Fig. 1. Paragraph [0034]-BORDER discloses as rapid motion is identified in first regions of the video frames and the assigned capture rate is increased for the rapid motion regions, second regions of the video frames are identified which have slow or no motion present and the capture rate is decreased for the regions with slow or no motion (wherein a region is a group of pixels).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of BORDER of receive one or more second enhancement parameters for a second region of the one or more regions; initiate the capture of the video based on the one or more second enhancement parameters; identify a plurality of pixels associated with the second region in each of the plurality of frames of the captured video; and apply the one or more second enhancement parameters to the identified plurality of pixels associated with the second region in each of the plurality of frames.
Wherein having WANG’s system of video quality enhancement having receive one or more second enhancement parameters for a second region of the one or more regions; initiate the capture of the video based on the one or more second enhancement parameters; identify a plurality of pixels associated with the second region in each of the plurality of frames of the captured video; and apply the one or more second enhancement parameters to the identified plurality of pixels associated with the second region in each of the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
Regarding claim 13, WANG in view of BORDER explicitly teach the system as claimed in claim 11,
WANG fails to explicitly teach wherein a field of view (FOV) of the reference image includes the one or more regions included in the video.
However, BORDER explicitly teaches wherein a field of view (FOV) of the reference image includes the one or more regions included in the video (Fig. 6, illustrates a field of view with one or more regions in a view (wherein #120 and #110 are regions. Paragraph [0022]-BORDER discloses FIG. 6 shows an illustration of a video frame 100 of a person fencing that has rapid motion in the region of the image 120 that contains the person's arm and the fencing foil, and slower motion in the region of the image 110 that contains the person's lower body. In an embodiment of the present invention, the local capture rate is increased in the regions (120) of the video frames 100 when the speed of motion relative to the exposure time in the region is above a motion blur threshold.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of BORDER of wherein a field of view (FOV) of the reference image includes the one or more regions included in the video.
Wherein having WANG’s system of video quality enhancement having wherein a field of view (FOV) of the reference image includes the one or more regions included in the video.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20120008832 A1), hereinafter referenced as WANG, in view of BORDER et al. (US 20100231738 A1), hereinafter referenced as BORDER, and further in view of CHOI et al. (US 20200322530 A1), hereinafter referenced as CHOI, and further in view of JUNG (US 20210035308 A1), hereinafter referenced as JUNG.
Regarding claim 4, WANG in view of BORDER explicitly teach the method as claimed in claim 1,
WANG further explicitly teach wherein the one or more first enhancement parameters include at least one of (Fig. 1. Paragraph [0018]-WANG discloses VQE module 16 may also selectively adjust parameters associated with the selected VQE function based on the estimated VQM value. In general, the parameters determine how much `enhancement` to apply to the ROI. For instance, in response to a VQM value that indicates a highly blurred image, the parameter associated with the sharpness enhancement function or super-resolution function would provide significant sharpness enhancement.):
WANG in view of BORDER fail to explicitly teach an exposure synthesis for enhancing a dynamic range of the captured video based on the one or more regions from the plurality of frames.
However, CHOI explicitly teaches an exposure synthesis (Fig. 1. Paragraph [0040]-CHOI discloses an electronic device according to various embodiments of the disclosure may include at least one of, for example, a smartphone, a tablet personal computer (PC), a mobile phone, a video phone, an electronic book reader (e-book reader), a desktop PC, a laptop PC, a netbook computer. Further in paragraph [0047]-CHOI discloses the electronic device 101 may include a processor 120, memory 130, an input device 150, a sound output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a subscriber identification module (SIM) 196, or an antenna module 197.) for enhancing a dynamic range of the captured video (Fig. 4A. Paragraph [0110]-CHOI discloses capturing an image may mean synthesizing or combining at least one image data obtained from at least one viewpoint to generate at least one image file and/or video file.) based on the one or more regions from the plurality of frames (Fig. 1. Paragraph [0176]-CHOI discloses the electronic device may synthesize different portions (e.g., a light region and a dark region) of a plurality of image data obtained by controlling the second camera based on different exposure values in order to acquire the image of which dynamic region is expanded. Alternatively, the electronic device may more greatly apply a gain of a relatively dark portion (e.g., a synthesis weight) than a gain of a relatively bright portion, from the plurality of image data obtained by controlling the second camera based on the same exposure value, thereby allowing data related to the dark portion to be acquired more clearly (wherein the synthesis weight is an exposure synthesis).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of CHOI of an exposure synthesis for enhancing a dynamic range of the captured video based on the one or more regions from the plurality of frames.
Wherein having WANG’s method of video quality enhancement having an exposure synthesis for enhancing a dynamic range of the captured video based on the one or more regions from the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and CHOI relate to improving the quality of a video frame/image, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while CHOI will identify a parameter to be used to control a camera at least based on an external electronic device, thereby improving the quality of an image captured in the camera of the electronic device. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and CHOI et al. (US 20200322530 A1), Paragraph [0011].
WANG in view of BORDER and further in view of CHOI fail to explicitly teach one or more motion blur parameters for synthesizing a silhouette of long exposure effect, or noise reduction parameters in one or more relatively static regions of the captured video based on one or more frames from the plurality of frames.
However, JUNG explicitly teaches one or more motion blur parameters for synthesizing a silhouette of long exposure effect (Fig. 1. Paragraph [0041]-JUNG discloses the motion vector calculation apparatus 11 may receive a long exposure image and a short exposure of a current frame from an image sensor 120, calculate a motion vector for respectively performing the blur correction with a long exposure image and a short exposure image of a previous frame, and performs blur correction using the motion vector (wherein the motion vector is one or more motion blur parameters and blur correction is synthesizing a silhouette of long exposure effect).), or
noise reduction parameters in one or more relatively static regions of the captured video based on one or more frames from the plurality of frames (Fig. 1. Paragraph [0048]-JUNG discloses when the gamma correction is performed, an image becomes brighter, but noise also increases. Therefore, 2-D noise reduction may be performed on the gamma-corrected image (wherein the gamma-corrected image is a relatively static region).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER and further in view of CHOI of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of JUNG of one or more motion blur parameters for synthesizing a silhouette of long exposure effect, or noise reduction parameters in one or more relatively static regions of the captured video based on one or more frames from the plurality of frames.
Wherein having WANG’s method of video quality enhancement having one or more motion blur parameters for synthesizing a silhouette of long exposure effect, or noise reduction parameters in one or more relatively static regions of the captured video based on one or more frames from the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and JUNG relate to analyzing specific regions of a video frame/image in order to improve the quality of the video frame/image, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while JUNG it is possible to increase the accuracy of calculation of a motion vector for blur correction. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and JUNG (US 20210035308 A1), Paragraph [0076].
Regarding claim 14, WANG in view of BORDER explicitly teach the system as claimed in claim 11,
WANG further explicitly teach wherein the one or more first enhancement parameters include at least one of (Fig. 1. Paragraph [0018]-WANG discloses VQE module 16 may also selectively adjust parameters associated with the selected VQE function based on the estimated VQM value. In general, the parameters determine how much `enhancement` to apply to the ROI. For instance, in response to a VQM value that indicates a highly blurred image, the parameter associated with the sharpness enhancement function or super-resolution function would provide significant sharpness enhancement.):
WANG in view of BORDER fail to explicitly teach an exposure synthesis for enhancing a dynamic range of the captured video based on the one or more regions from the plurality of frames.
However, CHOI explicitly teaches an exposure synthesis (Fig. 1. Paragraph [0040]-CHOI discloses an electronic device according to various embodiments of the disclosure may include at least one of, for example, a smartphone, a tablet personal computer (PC), a mobile phone, a video phone, an electronic book reader (e-book reader), a desktop PC, a laptop PC, a netbook computer. Further in paragraph [0047]-CHOI discloses the electronic device 101 may include a processor 120, memory 130, an input device 150, a sound output device 155, a display device 160, an audio module 170, a sensor module 176, an interface 177, a haptic module 179, a camera module 180, a power management module 188, a battery 189, a communication module 190, a subscriber identification module (SIM) 196, or an antenna module 197.) for enhancing a dynamic range of the captured video (Fig. 4A. Paragraph [0110]-CHOI discloses capturing an image may mean synthesizing or combining at least one image data obtained from at least one viewpoint to generate at least one image file and/or video file.) based on the one or more regions from the plurality of frames (Fig. 1. Paragraph [0176]-CHOI discloses the electronic device may synthesize different portions (e.g., a light region and a dark region) of a plurality of image data obtained by controlling the second camera based on different exposure values in order to acquire the image of which dynamic region is expanded. Alternatively, the electronic device may more greatly apply a gain of a relatively dark portion (e.g., a synthesis weight) than a gain of a relatively bright portion, from the plurality of image data obtained by controlling the second camera based on the same exposure value, thereby allowing data related to the dark portion to be acquired more clearly (wherein the synthesis weight is an exposure synthesis).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of CHOI of an exposure synthesis for enhancing a dynamic range of the captured video based on the one or more regions from the plurality of frames.
Wherein having WANG’s system of video quality enhancement having an exposure synthesis for enhancing a dynamic range of the captured video based on the one or more regions from the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and CHOI relate to improving the quality of a video frame/image, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while CHOI will identify a parameter to be used to control a camera at least based on an external electronic device, thereby improving the quality of an image captured in the camera of the electronic device. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and CHOI et al. (US 20200322530 A1), Paragraph [0011].
WANG in view of BORDER and further in view of CHOI fail to explicitly teach one or more motion blur parameters for synthesizing a silhouette of long exposure effect, or noise reduction parameters in one or more relatively static regions of the captured video based on one or more frames from the plurality of frames.
However, JUNG explicitly teaches one or more motion blur parameters for synthesizing a silhouette of long exposure effect (Fig. 1. Paragraph [0041]-JUNG discloses the motion vector calculation apparatus 11 may receive a long exposure image and a short exposure of a current frame from an image sensor 120, calculate a motion vector for respectively performing the blur correction with a long exposure image and a short exposure image of a previous frame, and performs blur correction using the motion vector (wherein the motion vector is one or more motion blur parameters and blur correction is synthesizing a silhouette of long exposure effect).), or
noise reduction parameters in one or more relatively static regions of the captured video based on one or more frames from the plurality of frames (Fig. 1. Paragraph [0048]-JUNG discloses when the gamma correction is performed, an image becomes brighter, but noise also increases. Therefore, 2-D noise reduction may be performed on the gamma-corrected image (wherein the gamma-corrected image is a relatively static region).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER and further in view of CHOI of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of JUNG of one or more motion blur parameters for synthesizing a silhouette of long exposure effect, or noise reduction parameters in one or more relatively static regions of the captured video based on one or more frames from the plurality of frames.
Wherein having WANG’s system of video quality enhancement having one or more motion blur parameters for synthesizing a silhouette of long exposure effect, or noise reduction parameters in one or more relatively static regions of the captured video based on one or more frames from the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and JUNG relate to analyzing specific regions of a video frame/image in order to improve the quality of the video frame/image, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while JUNG it is possible to increase the accuracy of calculation of a motion vector for blur correction. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and JUNG (US 20210035308 A1), Paragraph [0076].
Claims 5 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20120008832 A1), hereinafter referenced as WANG, in view of BORDER et al. (US 20100231738 A1), hereinafter referenced as BORDER, and further in view of SOMMERLADE et al. (US 20220383034 A1), hereinafter referenced as SOMMERLADE.
Regarding claim 5, WANG in view of BORDER explicitly teach the method as claimed in claim 1,
WANG further explicitly teaches wherein the segmenting the reference image into the one or more regions comprises (Fig. 1. Paragraph [0013]-WANG discloses object detection module 12 provides as an output a region of interest (ROI) that defines a localized portion of the received global image (wherein a localized portion region of interest is a segmented region).):
WANG in view of BORDER fail to explicitly teach segmenting the reference image into one or more regions according to one or more region masks.
However, SOMMERLADE explicitly teaches segmenting the reference image into one or more regions according to one or more region masks (Fig. 3A. Paragraph [0032]-SOMMERLADE discloses the segmentation processor 210 is configured to identify and/or classify target objects, such as a person's face, upper body, a hand, a hat, bottle, mug, pen, or other object. The segmentation processor 210 may identify “landmarks” within the target objects, provide locations of the landmarks to an object tracker (e.g., object tracker 310, FIG. 3A), determine labels and/or metadata for pixels of the input image, and generate a mask, as described below. The mask, such as mask 350, identifies which portions of the input image are a foreground and a background, to be separately processed by the foreground processor 220 and the background processor 230, respectively.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of SOMMERLADE of segmenting the reference image into one or more regions according to one or more region masks.
Wherein having WANG’s method of video quality enhancement having segmenting the reference image into one or more regions according to one or more region masks.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and SOMMERLADE relate to analyzing regions of video frames/images in order to improve the quality of the video frames/images, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, SOMMERLADE while aspects of the present disclosure are directed to improving image quality of a stream of input images. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and SOMMERLADE et al. (US 20220383034 A1), Paragraph [0003].
Regarding claim 15, WANG in view of BORDER explicitly teach the system as claimed in claim 11,
WANG further explicitly teaches wherein, for segmenting the reference image into the one or more regions (Fig. 1. Paragraph [0013]-WANG discloses object detection module 12 provides as an output a region of interest (ROI) that defines a localized portion of the received global image (wherein a localized portion region of interest is a segmented region).), the instructions, when executed by the one or more processors individually or collectively, cause the system to (Fig. 1. Paragraph [0024]-WANG discloses the present invention can also be embodied in the form of computer program code containing instructions embodied in computer readable medium, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by video processor 10, the processor becoming an apparatus for practicing the invention.):
WANG in view of BORDER fail to explicitly teach segment the reference image into one or more regions according to one or more region masks.
However, SOMMERLADE explicitly teaches segment the reference image into one or more regions according to one or more region masks (Fig. 3A. Paragraph [0032]-SOMMERLADE discloses the segmentation processor 210 is configured to identify and/or classify target objects, such as a person's face, upper body, a hand, a hat, bottle, mug, pen, or other object. The segmentation processor 210 may identify “landmarks” within the target objects, provide locations of the landmarks to an object tracker (e.g., object tracker 310, FIG. 3A), determine labels and/or metadata for pixels of the input image, and generate a mask, as described below. The mask, such as mask 350, identifies which portions of the input image are a foreground and a background, to be separately processed by the foreground processor 220 and the background processor 230, respectively.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of SOMMERLADE of segment the reference image into one or more regions according to one or more region masks.
Wherein having WANG’s system of video quality enhancement having segment the reference image into one or more regions according to one or more region masks.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and SOMMERLADE relate to analyzing regions of video frames/images in order to improve the quality of the video frames/images, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, SOMMERLADE while aspects of the present disclosure are directed to improving image quality of a stream of input images. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and SOMMERLADE et al. (US 20220383034 A1), Paragraph [0003].
Claims 6-7 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over WANG et al. (US 20120008832 A1), hereinafter referenced as WANG, in view of BORDER et al. (US 20100231738 A1), hereinafter referenced as BORDER, and further in view of SOMMERLADE et al. (US 20220383034 A1), hereinafter referenced as SOMMERLADE, and further in view of ADSUMILLI et al. (US 9355433 B1), hereinafter referenced as ADSUMILLI.
Regarding claim 6, WANG in view of BORDER and further in view of SOMMERLADE explicitly teach the method as claimed in claim 5,
WANG fails to explicitly teach wherein the identifying the plurality of pixels associated with the first region, comprises: identifying a plurality of pixels associated with each of the aligned one or more regions.
However, BORDER explicitly teaches wherein the identifying the plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the first region (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame (wherein capturing the different regions is identifying the pixels with the different regions).), comprises:
identifying a plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with each of the aligned one or more regions (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame.).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of BORDER of wherein the identifying the plurality of pixels associated with the first region, comprises: identifying a plurality of pixels associated with each of the aligned one or more regions.
Wherein having WANG’s method of video quality enhancement having wherein the identifying the plurality of pixels associated with the first region, comprises: identifying a plurality of pixels associated with each of the aligned one or more regions.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
WANG in view of BORDER fail to explicitly teach tracking the one or more regions in the plurality of frames based on the one or more region masks and one or more classes.
However, SOMMERLADE explicitly teaches tracking the one or more regions in the plurality of frames based on the one or more region masks and one or more classes (Fig. 3A-3B. Paragraph [0048]-SOMMERLADE discloses the object tracker 310 may provide a bounding box for tracked objects, the bounding box having an irregular shape or a regular shape (e.g., rectangle, circle, or ellipse), or a pixel mask where each pixel is associated with a classification, in various embodiments. In some embodiments, the object tracker 310 is an object classifier that recognizes and/or identifies one or more objects within the input image 340. In some embodiments, the object tracker 310 is configured to track faces and includes a facial recognition device or facial tracking device (not shown).);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of SOMMERLADE of tracking the one or more regions in the plurality of frames based on the one or more region masks and one or more classes.
Wherein having WANG’s method of video quality enhancement having tracking the one or more regions in the plurality of frames based on the one or more region masks and one or more classes.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and SOMMERLADE relate to analyzing regions of video frames/images in order to improve the quality of the video frames/images, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while SOMMERLADE aspects of the present disclosure are directed to improving image quality of a stream of input images. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and SOMMERLADE et al. (US 20220383034 A1), Paragraph [0003].
WANG in view of BORDER and further in view of SOMMERLADE fail to explicitly teach warping the tracked one or more regions; aligning the warped one or more regions; and.
However, ADSUMILLI explicitly teaches warping the tracked one or more regions (Fig. 1A-1B. Col. 4, Line [9-17]-ADSUMILLI discloses the image warp operation adjusts the shape and size of the image feature within each image such that the shape and size of the image feature is substantially similar across all images including the image feature. The amount of the adjustment to the shape and size of the image feature (or, the amount of warp applied to the image feature) is inversely proportional to the identified depth of the image feature, such that less warp is applied to image features far away from the cameras than is applied to image features close to the cameras (wherein the image features are the tracked one or more regions).);
aligning the warped one or more regions (Fig. 1A-1B. Col. 4, Line [19-20]- ADSUMILLI discloses the images can be aligned by aligning the warped image features.); and
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER and further in view of SOMMERLADE of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of ADSUMILLI of warping the tracked one or more regions; aligning the warped one or more regions; and.
Wherein having WANG’s method of video quality enhancement having warping the tracked one or more regions; aligning the warped one or more regions; and.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and ADSUMILLI analyze regions of video frames/images in order to improve the quality of the video frames/images, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while ADSUMILLI can partially reduce stitching artifacts within the stitched image 126B caused by the parallax error (for instance, by aligning the location within the overlap region 124B of each image feature for each of the images 120B and 122B). Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and ADSUMILLI et al. (US 9355433 B1), Col. 3, Line [15-34].
Regarding claim 7, WANG in view of BORDER and further in view of SOMMERLADE and further in view of ADSUMILLI explicitly teach the method as claimed in claim 6,
WANG fails to explicitly teach wherein the applying the one or more first enhancement parameters to the identified plurality of pixels associated to the first region, comprises: wherein the one or more enhancement parameters include the one or more first enhancement parameters and one or more second enhancement parameters.
However, BORDER explicitly teaches wherein the applying the one or more first enhancement parameters to the identified plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated to the first region (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame.), comprises:
wherein the one or more enhancement parameters include the one or more first enhancement parameters (Fig. 1, #125 called capture video image data with different frame rates in regions where different levels of motion are present. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshold (wherein the rate of capture is an enhancement parameter and a capture rate of the first region is the first enhancement parameter).) and one or more second enhancement parameters (Fig. 1. Paragraph [0026]-BORDER discloses in Step 115, capture rates for different regions within the video frames are assigned based on the determined relative speed of motion in the scene and the corresponding regions of the video frames. Regions in the video frames with slow or no motion, are assigned a relatively slower capture rate or maintained at the base capture rate, whereas, regions with rapid motion are assigned a relatively faster capture rate (wherein a slower or maintained rate of capture is a second enhancement parameter).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of BORDER of wherein the applying the one or more first enhancement parameters to the identified plurality of pixels associated to the first region, comprises: wherein the one or more enhancement parameters include the one or more first enhancement parameters and one or more second enhancement parameters.
Wherein having WANG’s method of video quality enhancement having wherein the applying the one or more first enhancement parameters to the identified plurality of pixels associated to the first region, comprises: wherein the one or more enhancement parameters include the one or more first enhancement parameters and one or more second enhancement parameters.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
WANG in view of BORDER and further in view of SOMMERLADE fail to explicitly teach applying one or more enhancement parameters to the identified plurality of pixels associated with the aligned one or more regions in each of the plurality of frames.
However, ADSUMILLI explicitly teaches applying one or more enhancement parameters to the identified plurality of pixels associated with the aligned one or more regions in each of the plurality of frames (Fig. 1A-1B. Col. 3, Line [50-55]- ADSUMILLI a image stitching algorithm can identify portions of each of two or more images representative of an overlap region between the two or more images, can align the identified portions of the images, and can average or feather the image data (such as the pixel color data) of the identified portions of the images to produce a stitched image (wherein the enhancement parameters are averaging or feathering image data, wherein the identified portions are the aligned regions, and wherein the frames are the two or more images).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER and further in view of SOMMERLADE of a method for capturing a video with enhanced quality, the method comprising: capturing a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video; segmenting the captured reference image into one or more regions; receiving one or more first enhancement parameters for a first region of the one or more regions with the teachings of ADSUMILLI of applying one or more enhancement parameters to the identified plurality of pixels associated with the aligned one or more regions in each of the plurality of frames.
Wherein having WANG’s method of video quality enhancement having applying one or more enhancement parameters to the identified plurality of pixels associated with the aligned one or more regions in each of the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and ADSUMILLI analyze regions of video frames/images in order to improve the quality of the video frames/images, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while ADSUMILLI can partially reduce stitching artifacts within the stitched image 126B caused by the parallax error (for instance, by aligning the location within the overlap region 124B of each image feature for each of the images 120B and 122B). Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and ADSUMILLI et al. (US 9355433 B1), Col. 3, Line [15-34].
Regarding claim 16, WANG in view of BORDER and further in view of SOMMERLADE explicitly teach the system as claimed in claim 15,
WANG further explicitly teaches wherein, the instructions, when executed by the one or more processors individually or collectively, cause the system to (Fig. 1. Paragraph [0024]-WANG discloses the present invention can also be embodied in the form of computer program code containing instructions embodied in computer readable medium, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by video processor 10, the processor becoming an apparatus for practicing the invention.):
WANG fails to explicitly teach for identifying the plurality of pixels associated with the first region, identify a plurality of pixels associated with each of the aligned one or more regions.
However, BORDER explicitly teaches for identifying the plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with the first region (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame (wherein capturing the different regions is identifying the pixels with the different regions).),
identify a plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated with each of the aligned one or more regions (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of BORDER of for identifying the plurality of pixels associated with the first region, identify a plurality of pixels associated with each of the aligned one or more regions.
Wherein having WANG’s system of video quality enhancement having for identifying the plurality of pixels associated with the first region, identify a plurality of pixels associated with each of the aligned one or more regions.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
WANG in view of BORDER fail to explicitly teach track the one or more regions in the plurality of frames based on the one or more region masks and one or more classes.
However, SOMMERLADE explicitly teaches track the one or more regions in the plurality of frames based on the one or more region masks and one or more classes (Fig. 3A-3B. Paragraph [0048]-SOMMERLADE discloses the object tracker 310 may provide a bounding box for tracked objects, the bounding box having an irregular shape or a regular shape (e.g., rectangle, circle, or ellipse), or a pixel mask where each pixel is associated with a classification, in various embodiments. In some embodiments, the object tracker 310 is an object classifier that recognizes and/or identifies one or more objects within the input image 340. In some embodiments, the object tracker 310 is configured to track faces and includes a facial recognition device or facial tracking device (not shown).);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of SOMMERLADE of track the one or more regions in the plurality of frames based on the one or more region masks and one or more classes.
Wherein having WANG’s system of video quality enhancement having track the one or more regions in the plurality of frames based on the one or more region masks and one or more classes.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and SOMMERLADE relate to analyzing regions of video frames/images in order to improve the quality of the video frames/images, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while SOMMERLADE aspects of the present disclosure are directed to improving image quality of a stream of input images. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and SOMMERLADE et al. (US 20220383034 A1), Paragraph [0003].
WANG in view of BORDER and further in view of SOMMERLADE fail to explicitly teach warp the tracked one or more regions; align the warped one or more regions; and.
However, ADSUMILLI explicitly teaches warp the tracked one or more regions (Fig. 1A-1B. Col. 4, Line [9-17]-ADSUMILLI discloses the image warp operation adjusts the shape and size of the image feature within each image such that the shape and size of the image feature is substantially similar across all images including the image feature. The amount of the adjustment to the shape and size of the image feature (or, the amount of warp applied to the image feature) is inversely proportional to the identified depth of the image feature, such that less warp is applied to image features far away from the cameras than is applied to image features close to the cameras (wherein the image features are the tracked one or more regions).);
align the warped one or more regions (Fig. 1A-1B. Col. 4, Line [19-20]- ADSUMILLI discloses the images can be aligned by aligning the warped image features); and
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER and further in view of SOMMERLADE of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of ADSUMILLI of warp the tracked one or more regions; align the warped one or more regions; and.
Wherein having WANG’s system of video quality enhancement having warp the tracked one or more regions; align the warped one or more regions; and.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and ADSUMILLI analyze regions of video frames/images in order to improve the quality of the video frames/images, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while ADSUMILLI can partially reduce stitching artifacts within the stitched image 126B caused by the parallax error (for instance, by aligning the location within the overlap region 124B of each image feature for each of the images 120B and 122B). Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and ADSUMILLI et al. (US 9355433 B1), Col. 3, Line [15-34].
Regarding claim 17, WANG in view of BORDER and further in view of SOMMERLADE and further in view of ADSUMILLI explicitly teach the system as claimed in claim 16,
WANG further explicitly teaches wherein, the instructions, when executed by the one or more processors individually or collectively, cause the system to (Fig. 1. Paragraph [0024]-WANG discloses the present invention can also be embodied in the form of computer program code containing instructions embodied in computer readable medium, such as floppy diskettes, CD-ROMs, hard drives, or any other computer-readable storage medium, wherein, when the computer program code is loaded into and executed by video processor 10, the processor becoming an apparatus for practicing the invention.):
WANG fails to explicitly teach for applying the one or more first enhancement parameters to the identified plurality of pixels associated to the first region, wherein the one or more enhancement parameters include the one or more first enhancement parameters and one or more second enhancement parameters.
However, BORDER explicitly teaches for applying the one or more first enhancement parameters to the identified plurality of pixels (Fig. 1. Paragraph [0024]-BORDER discloses multiple different capture rates (or frame rates) of regions or groups of pixels on an image sensor can be supported simultaneously on some types of image sensors such as CMOS image sensors (wherein a region is a group of pixels).) associated to the first region (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame.),
wherein the one or more enhancement parameters include the one or more first enhancement parameters (Fig. 1, #125 called capture video image data with different frame rates in regions where different levels of motion are present. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshold (wherein the rate of capture is an enhancement parameter and a capture rate of the first region is the first enhancement parameter).) and one or more second enhancement parameters (Fig. 1. Paragraph [0026]-BORDER discloses in Step 115, capture rates for different regions within the video frames are assigned based on the determined relative speed of motion in the scene and the corresponding regions of the video frames. Regions in the video frames with slow or no motion, are assigned a relatively slower capture rate or maintained at the base capture rate, whereas, regions with rapid motion are assigned a relatively faster capture rate (wherein a slower or maintained rate of capture is a second enhancement parameter).).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of BORDER of for applying the one or more first enhancement parameters to the identified plurality of pixels associated to the first region, wherein the one or more enhancement parameters include the one or more first enhancement parameters and one or more second enhancement parameters.
Wherein having WANG’s system of video quality enhancement having for applying the one or more first enhancement parameters to the identified plurality of pixels associated to the first region, wherein the one or more enhancement parameters include the one or more first enhancement parameters and one or more second enhancement parameters.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and BORDER relate to region-dependent video frame/image quality enhancement, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality. Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and BORDER et al. (US 20100231738 A1), Paragraph [0004].
WANG in view of BORDER and further in view of SOMMERLADE fail to explicitly teach apply one or more enhancement parameters to the identified plurality of pixels associated with the aligned one or more regions in each of the plurality of frames.
However, ADSUMILLI explicitly teaches apply one or more enhancement parameters to the identified plurality of pixels associated with the aligned one or more regions in each of the plurality of frames (Fig. 1A-1B. Col. 3, Line [50-55]- ADSUMILLI a image stitching algorithm can identify portions of each of two or more images representative of an overlap region between the two or more images, can align the identified portions of the images, and can average or feather the image data (such as the pixel color data) of the identified portions of the images to produce a stitched image (wherein the enhancement parameters are averaging or feathering image data, wherein the identified portions are the aligned regions, and wherein the frames are the two or more images).),
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of WANG in view of BORDER and further in view of SOMMERLADE of a system for capturing a video with enhanced quality, the system comprising: memory storing instructions; and one or more processors, wherein the instructions, when executed by the one or more processors individually or collectively, cause the system to: capture a reference image via a user equipment (UE), wherein the reference image is at least one frame of a plurality of frames of the video or an image associated with the video, segment the captured reference image into one or more regions, receive one or more first enhancement parameters for a first region of the one or more regions, with the teachings of ADSUMILLI of apply one or more enhancement parameters to the identified plurality of pixels associated with the aligned one or more regions in each of the plurality of frames.
Wherein having WANG’s system of video quality enhancement having apply one or more enhancement parameters to the identified plurality of pixels associated with the aligned one or more regions in each of the plurality of frames.
The motivation behind the modification would have been to obtain a video quality enhancement system that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both WANG and ADSUMILLI analyze regions of video frames/images in order to improve the quality of the video frames/images, wherein WANG provides selective, local enhancement to the defined ROI to improve the image quality in a way that will improve object recognition, while ADSUMILLI can partially reduce stitching artifacts within the stitched image 126B caused by the parallax error (for instance, by aligning the location within the overlap region 124B of each image feature for each of the images 120B and 122B). Please see WANG et al. (US 20120008832 A1), Paragraph [0008], and ADSUMILLI et al. (US 9355433 B1), Col. 3, Line [15-34].
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over of BORDER et al. (US 20100231738 A1), hereinafter referenced as BORDER, in view of CHIANG et al. (US 20200211201 A1), hereinafter referenced as CHAING.
Regarding claim 9, BORDER explicitly teaches a method for capturing a video with enhanced quality (Fig. 1. Paragraph [0031]-BORDER discloses this embodiment can be used to form an improved image with reduced blur wherein the capture cycle of different regions occurs simultaneously at different capture rates.), the method comprising:
providing at least one user interface for a user selection of one or more enhancement parameters for a selected region of the one or more regions of the first frame (Fig. 5, #560 called user interface. Paragraph [0025]-BORDER discloses the image processor 530 analyzes the image data, optionally in combination with inputs from the user through a user interface 560, to determine the capture parameters and the capture rate. Further in paragraph [0026]-BORDER discloses in Step 115, capture rates for different regions within the video frames are assigned based on the determined relative speed of motion in the scene and the corresponding regions of the video frames (wherein a capture rate is an enhancement parameter).); and
applying the one or more enhancement parameters to the selected region of the one or more regions of the first frame (Fig. 1. Paragraph [0026]-BORDER discloses in Step 105, the relative speed of motion of first regions within the scene with respect to second regions within the scene is determined by capturing and comparing two or more motion identification images, based on the changes in locations of objects within consecutive motion identification images and the time elapsed between the captures of the motion identification images. The relative speed of motion is compared to the exposure time for the video frame to determine the number of pixels of motion blur that would be caused by the motion if the video frame were to be captured at the base capture rate.) and a plurality of subsequent frames of the video during the video capture (Fig. 1. Paragraph [0026]-BORDER discloses in Step 125, images for the different regions of the video image are captured with the capture rates as assigned in Step 115 and stored or transmitted. As such, the image capture device causes the capture rate of a first region to be greater than the capture rate of a second region when the speed of motion within the first region, relative to the speed of motion within the second region, is above a predetermined relative speed or motion blur threshhold. The capture rate of the first region is selected to reduce the motion blur in each video frame.).
BORDER fails to explicitly teach segmenting a first frame of a plurality of frames of the video into one or more regions, while capturing the video.
segmenting a first frame of a plurality of frames of the video into one or more regions, while capturing the video (Fig. 4. Paragraph [0052]-CHIANG discloses a separate segmentation mask is generated for each individual 404, 406 depicted in the live video, where the each segmentation mask comprises facial feature vectors 118 of a facial region of each individual 404, 406, and where each facial feature vector 118 is generated from facial features of each individual (wherein a live video comprises of multiple video frames, wherein the currently viewed frame illustrated in fig. 4 is the first frame, and wherein the segmentation mask for each individual is a segmentation of the live video into one or more regions).);
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of BORDER of a method for capturing a video with enhanced quality, the method comprising: providing at least one user interface for a user selection of one or more enhancement parameters for a selected region of the one or more regions of the first frame; and applying the one or more enhancement parameters to the selected region of the one or more regions of the first frame and a plurality of subsequent frames of the video during the video capture with the teachings of CHIANG of segmenting a first frame of a plurality of frames of the video into one or more regions, while capturing the video.
Wherein having BORDER’s method of video quality enhancement having segmenting a first frame of a plurality of frames of the video into one or more regions, while capturing the video.
The motivation behind the modification would have been to obtain a video quality enhancement method that selectively enhances different regions of an image in order to improve the quality of the entire video frame/image. Since both BORDER and CHIANG analyze regions of video frames in order to improve the quality of the video, wherein BORDER it is therefore desirable to reduce blur of rapidly moving objects in video frames to improve video image quality, while CHAING the improved techniques disclosed herein allow a user to select one or more individuals in a live video to remain in the foreground while other individuals and/or objects that are not selected undergo background processing and are effectively hidden from view. Please see BORDER et al. (US 20100231738 A1), Paragraph [0004], and CHIANG et al. (US 20200211201 A1), Paragraph [0026].
Allowable Subject Matter
Claims 8, 10, and 18, along with their dependent claims, are therefrom objected to as being dependent upon rejected base claims, claim 1, 9, and 11, respectively, but would be allowable if rewritten in independent form including all of the limitations of the base claims and any intervening claims, once the specification and claim objections are overcome.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 8, the prior arts fail to explicitly teach determining a temporal loss and a perceptual loss of the identified plurality of overlapped regions; determining a set of blend weights to minimize a total loss based on the determined temporal loss and the determined perceptual loss, wherein the total loss includes the temporal loss and the perceptual loss; and feathering one or more region boundaries associated with the one or more regions by using the determined set of blend weights on discontinuities across the one or more regions, as claimed in claim 8.
Regarding claim 10, the prior arts fail to explicitly teach determining a set of blending weights for the identified plurality of overlapping pixels based on the determined temporal loss and the determined perceptual loss; and feathering the identified plurality of overlapping pixels based on the determined set of blending weights, as claimed in claim 10.
Regarding claim 18, the prior arts fail to explicitly teach determine a set of blend weights to minimize a total loss based on the determined temporal loss and the determined perceptual loss, wherein the total loss comprises the temporal loss and the perceptual loss, and feather one or more region boundaries associated with the one or more regions by using the determined set of blend weights on discontinuities across the one or more regions, as claimed in claim 18.
Conclusion
Listed below are the prior arts made of record and not relied upon but are considered pertinent to applicant’s disclosure.
EL-MALEH et al. (US 20120189168 A1) - The disclosure is directed to techniques for automatic segmentation of a region-of-interest (ROI) video object from a video sequence. ROI object segmentation enables selected ROI or "foreground" objects of a video sequence that may be of interest to a viewer to be extracted from non-ROI or "background" areas of the video sequence. Examples of a ROI object are a human face or a head and shoulder area of a human body. The disclosed techniques include a hybrid technique that combines ROI feature detection, region segmentation, and background subtraction. In this way, the disclosed techniques may provide accurate foreground object generation and low-complexity extraction of the foreground object from the video sequence. A ROI object segmentation system may implement the techniques described herein. In addition, ROI object segmentation may be useful in a wide range of multimedia applications that utilize video sequences, such as video telephony applications and video surveillance applications…Abstract, Fig. 1.
PERERA et al. (US 20090060373 A1) - Methods and computer program readable medium for restoring an image. The methods include the steps of selecting one or more frames followed by determining the regions of interest so that blurring effect is determined in the regions of interest using various techniques. The regions of interest are then deblurred and one of the deblurred regions of interest is then blended with the frame resulting in a restored frame…Abstract, Fig. 1.
JASINSKI et al. (US 20120201427 A1) - A method for determining a motion estimate, comprising: capturing at least two digital images of a scene at different capture times; designating one of the digital images as a reference digital image; designating a plurality of image regions; determining motion estimates for each image region by shifting the image regions within the non-reference digital images according to each of a plurality of spatial offsets relative to the image region within the reference digital image, computing merit function values for each spatial offset providing an indication of a difference between the image region in the reference digital image and the shifted image regions, computing a fitting function that provides an estimated merit function value as a function of spatial offset; and determining the motion estimate responsive to the determined non-integer spatial offset. A combined motion estimate is determined using weighting coefficients determined responsive to the shapes of the fitting functions…Abstract, Fig. 3 and 8.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ETHAN N WOLFSON whose telephone number is (571)272-1898. The examiner can normally be reached Monday - Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ETHAN N WOLFSON/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights