DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claim 6 is objected to because of the following informalities:
In line 2 of claim 6, applicant recites the phrase “the displacement parameter matrix” (emphasis added). However, applicant has not previously claimed a “displacement parameter matrix”, either in the prevous claim language of claim 6, or the parent claims to claim 6 (claims5 and 1). A more appropriate wording would be – a displacement parameter matrix”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-16 and 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, applicant recites the phrase “the enhanced images” (emphasis added to emphasize the plural language). However, applicant never previously claim the generation of multiple enhanced images used in reconstruction. The previous claim language recited “determining an enhanced image” – singular. It is unclear from the claim language or the specification whether only one generated enhanced image is used in the reconstruction step, or if more than one enhanced image is used in the reconstruction step. Therefore, applicant has failed to particularly point out and distinctly claim the subject matter that the inventor regards as the invention.
Claims 2-16 and 18 are rejected based on their dependency to claim 1.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1, 9, 12, 13, 15, 16, and 18 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Dai et al. (U.S. Publ. No. 2023/0237627).
Regarding claim 1, Dai discloses method and apparatus for forming a high dynamic range image from multiple lower dynamic range images. More specifically and as it relates to the applicant’s claims, Dai discloses a method for reconstructing an HDR image (see paragraph 0026 where the disclosed method is used in generating an HDR image), wherein the method comprises:
acquiring a plurality of original images with same photographing scene and different exposure degrees (see paragraph 0073 where the camera captures multiple images having different exposures);
screening out a reference image from the plurality of original images (paragraph 0027 and 0041 where a reference image is set; also see paragraph 0051 and Figure 4 where L2 is set as the reference frame), performing feature alignment processing on remaining original images according to the reference image (see Figure 4 and paragraph 0052 where feature maps are generated from various the image inputs; this generation of feature maps is interpreted as the alignment processing. This aligns with applicant disclosure of how feature alignment is performed as disclosed in their spec – see 0139-0142 of applicant’s specification), and obtaining displacement images of the remaining original images (see Figure 4 and paragraph 0052; the output of the attention modules (405 and 406) is interpreted to be the displacement images, since it done identically to how applicant discloses it being done – see paragraphs 0139-042 of applicants specification);
determining an enhanced image according to the reference image and the displacement images of the remaining original images (the enhanced image is interpreted as the output of the discrete wavelet transform processing performed in steps 407-411), wherein the enhanced image is obtained by performing image enhancement processing on a fused image (the fused image is interpreted as the concatenated feature map images, 406) after a down-sampling operation (the discrete wavelet transform (DCT) is a downsampling operation), and the fused image is obtained by performing feature fusion on the reference image and the displacement images of the remaining original images (the fused image is interpreted as the concatenated feature map images, 406); and
reconstructing the HDR images corresponding to the plurality of original images according to the enhanced images (see Figure 3 and paragraph 0049 where the enhanced image is reconstructed to form an HDR image).
As for claim 9, Dai discloses performing image enhancement processing on a fused image after a down-sampling operation comprises:
performing the down-sampling operation on the fused image to obtain a down-sampled image (see Figure 4 and paragraph 0053 where image 406 is the fused image that is input into DWT step 407, which is inherently a downsampling operation); and
performing image enhancement processing on the down-sampled image, wherein the image enhancement processing is configured for aligning similar features in the down-sampled image and enhancing a feature characterizing an image detail in the down-sampled image (see Figure 4 the enhanced image is interpreted as the output of the discrete wavelet transform processing performed in steps 407-411; also see Figure 53 where the blocks 408-411 merge features into feature map 411; furthermore, the DWT operation enhances feature characterizing image detail in images – including edges).
With regard to claim 12, Dai discloses that screening out a reference image from the plurality of original images comprises: selecting an original image with a middle exposure degree as the reference image from the plurality of original images (also see paragraph 0051 and Figure 4 where L2 is set as the reference frame).
As for claim 13, Dai discloses that acquiring a plurality of original images with same photographing scene and different exposure degrees comprises: in response to a photographing instruction of a user, successively photographing the plurality of original images with different exposure degrees by a camera assembly for a same photographing scene. See paragraph 0073.
Regarding claim 15, claim 15 is considered an apparatus claim for carrying out the method of claim 1. Please see the discussion of claim 1 above. Furthermore, Dai discloses a terminal (imaging device, 701; see Figure 7 and paragraph 0074) for reconstructing an HDR image, wherein the terminal comprises a processor (processor, 704; see Figure 7 and paragraph 0074) and a memory , the memory (memory, see paragraph 0075) is configured for storing a program executable by the processor (see paragraph 0075 where code is executed by the processor for carrying out the method), and the processor is configured for reading the program in the memory and performing steps of the method (see paragraph 0075 where code is executed by the processor for carrying out the method).
As for claim 16, claim 16 is considered an apparatus claim for carrying out the method of claim 1. Please see the discussion of claim 1 above. Furthermore, Dai discloses an electronic device for reconstructing an HDR image (imaging device, 701; see Figure 7 and paragraph 0074), wherein the electronic device comprises a camera unit (camera; see paragraph 0073) and a control circuit (processor, 704 and memory; see Figure 7 and paragraphs 0074 and 0075), wherein the camera unit is configured for acquiring original images with different exposure degrees (see paragraph 0073); and the control circuit comprises a processor (processor, 704; see Figure 7 and paragraph 0074) and a memory (memory, see paragraph 0075), the memory is configured for storing a program executable by the processor, and the processor is configured for reading the program in the memory and performing steps of the method (see paragraph 0075 where code stored in memory is executed by the processor for carrying out the method).
With regard to claim 18, claim 18 is considered an computer readable medium claim for carrying out the method of claim 1. Please see the discussion of claim 1 above. Furthermore, Dai discloses a non-transitory computer storage medium (memory, see paragraph 0075), storing computer programs (code; see paragraph 0075), wherein the computer programs, when executed by a processor, implement steps of the method (see paragraph 0075 where code stored in memory is executed by the processor for carrying out the method).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7 is rejected under 35 U.S.C. 103 as being unpatentable over Dai et al. (U.S. Publ. No. 2023/0237627) in view of Gao et al. (U.S. Publ. No. 2020/0374525).
Regarding claim 7, as mentioned above in the discussion of claim 1, Dai disclose all of the limitations of claim 1. Dai however, fails to explicitly disclose that a sampling multiple of the down-sampling operation is determined according to a calculation force of a hardware platform. Gao, on the other hand discloses that it is well known in the art to adjust a downsampling ratio based on the capabilities of the processor doing the downsampling. See paragraphs 0140 and 0142. One of ordinary skill in the art would recognize the benefits of adjusting the amount of downsampling based on a processors capability – including faster processing of the data, and preventing the overload of the processor. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Dai such that the sampling multiple of the down-sampling operation is determined according to a calculation force of a hardware platform, for the reasons stated above.
Claim(s) 11 is rejected under 35 U.S.C. 103 as being unpatentable over Dai et al. (U.S. Publ. No. 2023/0237627) in view of Kwong et al. (U.S. Publ. No. 2023/0179871).
Regarding claim 11, as mentioned above in the discussion of claim 1, Dai disclose all of the limitations of claim 1. Dai however, fails to explicitly disclose that reconstructing the HDR images corresponding to the plurality of original images according to the enhanced images comprises: performing dimension reduction processing on the enhanced image through a plurality of convolution layers to obtain the HDR image. Kwong, on the other hand, discloses that it is well known in the art to perform dimesnion reduction processing using convolution layers to obtain an HDR image during a reconstruction operation. In particular, Kwong discloses a dimension reduction step during reconstruction of a final HDR image. See paragraph 0064. Kwong discloses that this feature helps the network learn a better downsample function. See paragraph 0064. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Dai to perform dimsnion reduction processing on the enhanced image of Dai using convolution layers to obtain the HDR image so that the network can learn a better downsample function.
Claim(s) 14 is rejected under 35 U.S.C. 103 as being unpatentable over Dai et al. (U.S. Publ. No. 2023/0237627).
Regarding claim 14, as mentioned above in the discussion of claim 1, Dai disclose all of the limitations of claim 1. Dai however, fails to explicitly disclose after reconstructing the HDR images corresponding to the plurality of original images according to the enhanced images, the method further comprises: displaying a reconstructed HDR image on a display. However, Official Notice is taken as to the fact that images are commonly displayed on a display of a device after performing HDR processing. One of ordinary skill in the art would recognize the benefits of including a display on the camera of Dai – including giving the user the ability to view the improved image. Therefore, it have been obvious to one of ordinary skill in the art before the effective filing date of the invention to modify Dai to include a display for displaying the reconstructed HDR image.
Allowable Subject Matter
Claims 2-6, 8, and 10 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims and if the rejections under 35 U.S.C. 112(b) listed above are overcome.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest after obtaining displacement images of the remaining original images and before determining an enhanced image according to the reference image and the displacement images of the remaining original images, the method further comprises: performing at least one time of down-sampling feature alignment processing, wherein each time of down-sampling feature alignment processing performs the following steps of: performing the down-sampling operation on a reference feature image and the displacement images of the remaining original images to obtain a reference feature image and a displacement image after the down-sampling operation, wherein the reference feature image is obtained by performing feature extraction on the reference image; and according to the reference feature image after the down-sampling operation, performing feature alignment processing on the displacement images after the down-sampling operation to obtain a displacement image after the feature alignment processing of the current time.
As for claim 3, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest that after determining an enhanced image according to the reference image and the displacement images, and before reconstructing the HDR images corresponding to the plurality of original images according to the enhanced images, the method further comprises: performing at least one time of image enhancement operation; wherein each time of image enhancement operation performs the following step of taking the enhanced image determined last time as the reference image of a current time, and determining the enhanced image of the current time according to the reference image of the current time and the displacement image.
With regard to claim 4, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest performing feature alignment processing on remaining original images according to the reference image, and obtaining displacement images of the remaining original images comprises: according to feature similarity between the reference image and each remaining original image, determining a displacement parameter matrix corresponding to the remaining original image; and performing displacement on features of the remaining original images according to the displacement parameter matrix to obtain the displacement images.
With regard to claim 5, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest performing feature alignment processing on remaining original images according to the reference image, and obtaining displacement images of the remaining original images comprises: performing feature extraction on the reference image to obtain a reference feature image, and performing feature extraction on each remaining original image to obtain an original feature image; respectively merging the reference feature image and each original feature image to obtain a first merged image corresponding to each original feature image; and inputting the first merged image and a corresponding original feature image into an attention network, and outputting the displacement image of the original feature image.
Regarding claim 8, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest that the fused image is determined in the following manner of merging the reference feature image obtained by performing feature extraction on the reference image and the displacement images of the remaining original images to obtain a second merged image; and reducing a dimension of the second merged image via a convolution layer to obtain the fused image.
As for claim 10, the primary reason for indication of allowable subject matter is that the prior art fails to teach or reasonably suggest inputting the down-sampled image into an image enhancement network and outputting a network image, wherein the image enhancement network is configured for aligning similar features in the down-sampled image, and enhancing the feature characterizing the image detail in the down-sampled image; and performing an up-sampling operation on the network image to obtain the enhanced image.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHN M VILLECCO whose telephone number is (571)272-7319. The examiner can normally be reached Mon-Thurs 6:00 AM-4:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JOHN VILLECCO/Supervisory Patent Examiner, Art Unit 2661