DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Amendments
Acknowledgment of receiving amendments to the claims, which were received by the Office on 09/17/2025.
Response to Arguments
Applicant's arguments filed 09/17/2025 have been fully considered but they are not persuasive.
In that remarks, applicant argues in substance:
Applicant argues: “Applicant submits that Hinkel does not disclose the subject matter of independent claim 1 as amended, in particular the features of "a camera imager with a noncircular aperture, the camera imager configured to capture image content at a depth of field in an image" and "to adaptively adjust a perceived depth of field to sharpen at least a portion of the image content in the image based at least in part on the identifiable characteristic resulting from the noncircular aperture of the camera imager."
Hinkel describes capturing multiple images, each with a different shaped aperture and at different times in an effort to detect bokeh artifacts (blurred regions) as an indication of motion of objects in a captured scene. (Hinkel, [0024], [0025], [0030]). As the Office indicates, an objective of Hinkel is to capture more than one image with different shaped apertures, citing Hinkel Fig.5, Steps 518, 520 to detect first shape bokeh artifacts in a first image and second shape bokeh artifacts in a second image.GA p. 3.
Contrary to Hinkel, Applicant describes and claims to capture image content in a single image with just one noncircular aperture of a camera imager. Notably, Hinkel does not disclose "a camera imager with a noncircular aperture, the camera imager configured to capture image content at a depth of field in an image" or "to adaptively adjust a perceived depth of field to sharpen at least a portion of the image content in the image based at least in part on the identifiable characteristic resulting from the noncircular aperture of the camera imager" as recited in claim 1.”
Examiner’s Response: Examiner respectfully disagrees. Claim language does not limit what may be considered “an image”.
Hinkel recites in paragraph 0025:
[0025] The present disclosure improves the field of photography (e.g., the art, science and practice of creating durable images by recording light) by improving clarity in photographs captured by an image capture device, such as a camera. Clarity in images is improved by obtaining additional information about a scene, such as the information about movement of the camera during the image capture event (e.g., caused by camera shake etc.), information about objects in motion within the scene at the time of the exposure, depth information about objects within the scene at the time of the exposure, and information about details in high intensity (i.e., bright areas) and areas in shadow. Some of this information may be obtained by changing a shape of aperture during the exposure, scanning for the occurrence of bokeh artifacts in the captured image corresponding to the shape of the aperture before the change and corresponding to the shape of the aperture after the change. When using a charge-coupled image capture device, parts of the captured image may be extracted at different times during the exposure; for example, in a charge-coupled device having arrays of N rows (with N being a positive integer), data collected by even-numbered rows of the charge-coupled device may be retrieved at a first stage in the exposure when the aperture is in a first shape, and, at a later stage when the aperture is in a second shape, data collected by odd-numbered rows of the charge-coupled device may be received. Image data retrieved during the first stage may be compared with image data retrieved during the later stage to glean information about the scene for use in providing clarity to a final image. For example, by comparing bokeh artifacts occurring in the first stage image data with corresponding bokeh artifacts occurring in the later stage image data, data about the motion (e.g., speed and direction) of the camera and/or data about the motion (e.g., speed and direction) of objects in the scene may be determined.
Hinkel states a “captured image” may comprise image data collected by even-numbered rows and data collected by odd-numbered rows which are retrieved at different stages. With reference to Figure 5 of Hinkel, at Step 504, “an image capture exposure begins” (Hinkel, Paragraph 0056). At step 506, “a first set of rows of the charge-coupled device sensor array of the image capture device is read” to produce a first set of image data. (Hinkel, Paragraph 0057 and 0061). At step 510, “a second set of rows of the charge-coupled device sensor array of the image capture device is read” to produce a second set of image data (Hinkel, Paragraph 0058 and 0061). At step 512, the image capture period is over (Hinkel, Paragraph 0059).
Therefore, Steps 504-512 of Hinkel may be seen as capturing “an image”. The image containing a first set of image data and a second set of image data.
Applicant argues: “Claim 3 recites "to adaptively adjust the perceived depth of field, the content manager is configured to utilize optical properties of at least a camera lens of the camera device used to capture the image content." The Office cites Hinkel [0052] for the recited features of dependent claim 3. GA p. 4. However, Hinkel does not disclose using optical properties of a camera lens to deblur an image. Rather, Hinkel [0052] only describes that motion blur can occur when the camera moves during the exposure period, or when an object that is being captured moves during the exposure period. There is no indication in Hinkel of taking into account the feature of "at least a camera lens of the camera device used to capture the image content" as recited in claim 3.”
Examiner’s Response: Examiner respectfully disagrees. Claim language does not limit what may be considered to be how the content manager utilizes optical properties of at least a camera lens. Optical properties of the lens are used in capturing the image which affect the focus or blur of the objects (Hinkel, Paragraph 0052). The image is used when performing deblurring. Therefore, optical properties of the lens are considered to be used when adjust the perceived depth of field.
Applicant argues: “Applicant further submits that it would not have been obvious to combine Hinkel with any of the cited references Lee, Wadhwa, Sun, Hiasa, or Alon because there is no indication in Hinkel to use any type of machine learning or convolutional network to adaptively adjust the depth of field of image content in an image. As noted above, Hinkel describes capturing multiple images each with a different shaped aperture and at different times in an effort to detect motion blur in a captured scene. Notably, Hinkel seeks to mitigate the motion blur based on the captured multiple images, rather than by using any type of machine learning network. There is no provided evidence to explain why Hinkle suffers from a lack of using a machine learning network. Accordingly, the alleged motivation to combine Hinkle with any of the cited references lacks rational underpinnings because Hinkle does not need or rely on a machine learning network. Further, there is no indication in any of the cited references Lee, Wadhwa, Sun, Hiasa, or Alon of "a camera imager with a noncircular aperture" as recited in claim 1, which precludes any type of obvious combination with Hinkel.”
Examiner’s Response: Examiner respectfully disagrees. It would have been obvious to one of ordinary skill in the art to modify the invention of Hinkel with the method of deblurring using a machine learning model as seen in Lee to enhancing a quality of an image and, more particularly, to removing or mitigating a defocus blur. It would have been obvious to one of ordinary skill in the art to modify the combination of Hinkel and Lee with the teaching of using training data having characteristics of images captured with a mobile phone camera as seen in Wadhwa since the images the machine learning model may be used for correcting images captured with a mobile phone camera (as seen in Hinkel). It would have been obvious to modify the combination of Hinkel and Lee with the teaching of using training images that have sharp foreground content and blurry background content as seen in Sun with the teaching of using images having both sharp foreground content and blurry background content as seen in Sun since it is a known type of blurred image to be compared with a in focus image when used for training a machine learning model and would produce similar and expected results. It would have been obvious to modify the combination of Hinkel and Lee with the teaching of using the training image pair as seen in Hiasa since it is a known method of producing ground truth and training images and would provide similar and expected results to training to deblur images. It would have been obvious to modify the invention of Hinkel with the teaching processing separate blocks and stitching the blocks together to form the sharpened image to account for varying PSF over the image field.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-5 and 12-15 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Hinkel et al. (US 2018/0048820 A1).
Regarding claim 1, Hinkel et al. (hereafter referred as Hinkel) teaches a camera device (Hinkel, Paragraph 0025), comprising:
a camera imager (Hinkel, Fig. 3, image sensor 302, Paragraph 0041) with a noncircular aperture (Hinkel, Figs. 1-2, Paragraphs 0028 and 0038-0040, The aperture is octagonal in figure 1.), the camera imager configured to capture image content at a depth of field in an image (Hinkel, Paragraph 0034, The depth of field is determined by the aperture.), the image content having an identifiable characteristic resulting from the noncircular aperture of the camera imager (Hinkel, Fig. 4, bokeh effects 412A/B, Paragraphs 0025, 0030 and 0046); and
a content manager implemented at least partially in computer hardware (Hinkel, Paragraphs 0130-0131, “processor, such as a central processing unit (CPU)”) and configured to adaptively adjust a perceived depth of field to sharpen at least a portion of the image content in the image based at least in part on the identifiable characteristic resulting from the noncircular aperture of the camera imager (Hinkel, Fig. 5, Steps 518-520, Paragraphs 0048 and 0062-0063, Deblurring the image is considered to be increasing (adjusting) a perceived depth of field).
Claim 12 is rejected for the same reasons as claim 1.
Regarding claim 2, Hinkel teaches the camera device of claim 1 (see claim 1 analysis), wherein, to adaptively adjust the perceived depth of field, the content manager is configured to increase the perceived depth of field to compensate for the identifiable characteristic resulting from the noncircular aperture (Hinkel, Fig. 5, Steps 518-520, Paragraphs 0048 and 0062-0063, Deblurring the image is considered to be increasing a perceived depth of field).
Claim 13 is rejected for the same reasons as claim 2.
Regarding claim 3, Hinkel teaches the camera device of claim 1 (see claim 1 analysis), wherein, to adaptively adjust the perceived depth of field, the content manager is configured to utilize optical properties of at least a camera lens of the camera device used to capture the image content (Hinkel, Paragraphs 0052, Optical properties of the lens are used in capturing the image. The image is used when performing deblurring. Therefore, optical properties of the lens are used when adjust the perceived depth of field.).
Claim 14 is rejected for the same reasons as claim 3.
Regarding claim 4, Hinkel teaches the camera device of claim 1 (see claim 1 analysis), wherein the identifiable characteristic resulting from the noncircular aperture of the camera imager is a distortion of at least the portion of the image content (Hinkel, Fig. 4, bokeh effects 412A/B, Paragraphs 0025, 0030 and 0046).
Regarding claim 5, Hinkel teaches the camera device of claim 4 (see claim 4 analysis), wherein, to adaptively adjust the perceived depth of field, the content manager is configured to increase the perceived depth of field to compensate for the distortion of at least the portion of the image content (Hinkel, Fig. 5, Steps 518-520, Paragraphs 0048 and 0062-0063, Deblurring the image is considered to be increasing a perceived depth of field).
Regarding claim 15, Hinkel teaches the method of claim 12 (see claim 12 analysis), further comprising: increasing the perceived depth of field to compensate for distortion of at least the portion of the image content (Hinkel, Fig. 5, Steps 518-520, Paragraphs 0048 and 0062-0063, Deblurring the image is considered to be increasing a perceived depth of field), wherein the identifiable characteristic resulting from the noncircular aperture of the camera imager is the distortion of at least the portion of the image content (Hinkel, Fig. 4, bokeh effects 412A/B, Paragraphs 0025, 0030 and 0046).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 6-7, 9, 16 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hinkel et al. (US 2018/0048820 A1) in view of Lee et al. (US 2023/0196520 A1).
Regarding claim 6, Hinkel teaches the camera device of claim 1 (see claim 1 analysis. However, Hinkel does not teach wherein the content manager is a machine learning model configured to at least one of correct a blur or reduce noise of at least the portion of the image content.
In reference to Lee et al. (hereafter referred as Lee), Lee teaches content manager is a machine learning model (Lee, Figs. 2 and 6, Paragraphs 0085 and 0097-0098) configured to at least one of correct a blur (Lee, Paragraph 0003) or reduce noise of at least the portion of the image content (Lee, Paragraphs 0085-0087 and 0101, Lee teaches a machine learning model for deblurring an image where blur depends on an aperture shape and a lens design of the camera.).
These arts are analogous since they are both related on deblurring images where blur depends on an aperture shape and a lens design of the camera. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Hinkel with the method of deblurring using a machine learning model as seen in Lee to enhancing a quality of an image and, more particularly, to removing or mitigating a defocus blur.
Claims 16 and 18-20 are rejected for the same reasons as claim 6.
Regarding claim 7, the combination of Hinkel and Lee teaches the camera device of claim 6 (see claim 6 analysis), wherein the machine learning model is trained to learn an inverse image of the image content (Lee, Fig. 7, Paragraphs 0048-0053, 0101-0103, The image content is the blurred image. The machine learning model is trained to inverse the blur of the blurred image, that is, trained to “learn an inverse image.”).
Regarding claim 9, the combination of Hinkel and Lee teaches the camera device of claim 6 (see claim 6 analysis), wherein the machine learning model is trained with one or more training images that have sharp foreground content (Lee, Paragraph 0101, Images that have sharp foreground content are “ground truth sharp” images.) and blurry background content (Lee, Paragraph 0101, Images that have blurry background content are “defocus blur” images.), and an output of the content manager is sharpened image content (Hinkel, Fig. 5, Steps 518-520, Paragraphs 0048 and 0062-0063, Lee, Fig. 2, deblurred sharp image 99).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hinkel et al. (US 2018/0048820 A1) in view of Lee et al. (US 2023/0196520 A1) in view of Wadhwa et al. (US 2021/0183089 A1).
Regarding claim 8, the combination of Hinkel and Lee teaches the camera device of claim 6 (see claim 6 analysis), wherein the images are captured with a mobile phone camera having an asymmetric aperture (Hinkel, Paragraphs 0098 and 0126).
However, the combination of Hinkel and Lee does not teach wherein the machine learning model is trained with one or more training images converted to have characteristics of images captured with a mobile phone camera having an asymmetric aperture.
In reference to Wadhwa et al. (hereafter referred as Wadhwa), Wadhwa teaches wherein the machine learning model is trained with one or more training images converted to have characteristics of images captured with a mobile phone camera having varying apertures (Wadhwa, Paragraph 0068).
These arts are analogous since they are both related to imaging devices and image blur. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Hinkel and Lee with the teaching of using training data having characteristics of images captured with a mobile phone camera as seen in Wadhwa since the images the machine learning model may be used for correcting images captured with a mobile phone camera (as seen in Hinkel). Further, it would have been obvious to train the machine learning model with images captured with a mobile phone camera having an asymmetric aperture since the machine learning model in the combination of Hinkel and Lee is used for images that are captured with a mobile phone camera having an asymmetric aperture.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hinkel et al. (US 2018/0048820 A1) in view of Lee et al. (US 2023/0196520 A1) in view of Sun (US 2022/0398704 A1).
Alternatively, regarding claim 9, the combination of Hinkel and Lee teaches the camera device of claim 6 (see claim 6 analysis), wherein an output of the content manager is sharpened image content (Hinkel, Fig. 5, Steps 518-520, Paragraphs 0048 and 0062-0063).
However, the combination of Hinkel and Lee does not teach wherein the machine learning model is trained with one or more training images that have sharp foreground content and blurry background content.
In reference to Sun, sun teaches wherein the machine learning model is trained with one or more training images that have sharp foreground content and blurry background content (Sun, Paragraphs 0042 and 0052).
These arts are analogous since they are both related to imaging devices and image blur. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Hinkel and Lee with the teaching of using training images that have sharp foreground content and blurry background content as seen in Sun with the teaching of using images having both sharp foreground content and blurry background content as seen in Sun since it is a known type of blurred image to be compared with a in focus image when used for training a machine learning model and would produce similar and expected results. That is, the in focus image and shallow focus image pair of Sun is equivalent to the “ground truth sharp” image and “defocus blur” image of Lee (Lee, Paragraph 0101).
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hinkel et al. (US 2018/0048820 A1) in view of Lee et al. (US 2023/0196520 A1) in view of Hiasa (US 2020/0388014 A1).
Regarding claim 10, the combination of Hinkel and Lee teaches the camera device of claim 6 (see claim 6 analysis). However, the combination of Hinkel and Lee does not teach wherein the machine learning model is trained with a training image pair, a first image of the training image pair having high resolution, low noise, and a large depth of field, and a second image of the training image pair is generated by blurring and adding noise to the first image.
In reference to Hiasa, Hiasa teaches wherein the machine learning model is trained with a training image pair, a first image of the training image pair having high resolution, low noise, and a large depth of field (Hiasa, Paragraphs 0046 and 0051, The first image is a “ground truth image with high resolution”. The ground truth image does not include the blurs (noise). Blurs include defocus (Paragraph 0035), an image without defocus is considered to be an image with a large depth of field.) and a second image of the training image pair is generated by blurring and adding noise to the first image (Hiasa, Paragraph 0051-0052, The blurs (noise) are applied to generate the training image.).
These arts are analogous since they are all related to imaging devices and image deblurring. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the combination of Hinkel and Lee with the teaching of using the training image pair as seen in Hiasa since it is a known method of producing ground truth and training images and would provide similar and expected results to training to deblur images.
Claim(s) 11 and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hinkel et al. (US 2018/0048820 A1) in view of Alon et al. (US 2006/0256226 A1).
Regarding claim 11, Hinkel teaches the camera device of claim 1 (see claim 1 analysis), wherein, to adaptively adjust the perceived depth of field, the content manager is configured to segregate the image content into region slices (Hinkel, Fig. 4, Paragraph 0043, Region slices are odd and even fields.), separately process each region slice to compensate for the identifiable characteristic resulting from the noncircular aperture (Hinkel, Fig. 5, Step 518, analysis for occurrences of bokeh artifacts is processing for compensating the identifiable characteristic.).
However, Hinkel does not teach stitching compensated region slices back together to generate a sharpened image of the image content.
In reference to Alon et al. (hereafter referred as Alon), Alon teaches wherein, to adaptively adjust the perceived depth of field, the content manager is configured to segregate the image content into region slices (Alon, Paragraph 0085),
separately process each region slice to compensate for the identifiable characteristic (Alon, Paragraphs 0008 and 0085), and
stitch compensated region slices back together to generate a sharpened image of the image content (Alon, Paragraph 0008 and 0085).
These arts are analogous since they are both related to imaging devices and image deblurring. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Hinkel with the teaching processing separate blocks and stitching the blocks together to form the sharpened image to account for varying PSF over the image field.
Claim 17 is rejected for the same reasons as claim 11.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WESLEY JASON CHIU whose telephone number is (571)270-1312. The examiner can normally be reached Mon-Fri: 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Twyler Haskins can be reached at (571) 272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WESLEY J CHIU/Examiner, Art Unit 2639
/TWYLER L HASKINS/Supervisory Patent Examiner, Art Unit 2639