DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 21 rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1, 3, 4, 6, 7, 10, 11, 13, 14, 16, and 17 of U.S. Patent No. 12,327,404. Although the claims at issue are not identical, they are not patentably distinct from each other because the claims of the current application are broader than the claims of the conflicting patent, as shown in the table below:
Current application 19/046901
Conflicting patent 12,327,404
21
1
22
1
23
3
24
3
25
1
26
4
27
6
28
7
29
7
30
10
31
10
32
7
33
11
34
6
35
13
36
14
37
13
38
6
39
16
40
17
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “preprocessing module” and “object detection module” in claim 21.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 28-35, and 37-40 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou et al. (Deep Neural Network Based Salient Object Detection with Image Enhancement, 17 November 2018, LNTCS, Vol. 11304, Pages 444-453), hereinafter “Zhou”, in view of Dulski et al. (Evaluation of hardware implementation of the infrared image enhancement algorithm, 2012, QIRT, pages 1-9), hereinafter “Dulski”, in view of Conti et al. (A Regularized Deep Learning Approach for Image De-Blurring, 2017, IML ’17, Pages 1-5), hereinafter “Conti”.
Claim 28 is met by the combination of Zhou, Dulski, and Conti, wherein
Zhou discloses:
An apparatus (See the Abstract.) comprising:
a processor (See page 449: “We train and test all these datasets on a NVIDIA GeForce GTX1080Ti GPU, with in total 200000 training iterations.”)…configured to process…a second ANN (See the saliency prediction network in Fig. 1.),
wherein the second ANN is configured to perform object detection function to the image to detect at least one object included in the image (See page 445: “Inspired by the outstanding performance made by FCN, meanwhile to make use of the priors, we proposed an end-to-end model for salient object detection.” Note: as currently recited, the examiner interprets this claim so that the output of the “first ANN” is not necessarily required to be the input of the “second ANN”. In other words, the “first ANN” and “second ANN” receive a copy of the “image” and function independently. In contrast, claim 1 requires the “first ANN” to pass a deblurred and denoised image to the “second ANN”.).
Zhou does not disclose the following; however, Dulski discloses:
a processor, implemented into a form of an application specific integrated circuit (ASIC) (See page 5: “Complex algorithms for real time image processing are often executed by specially designed ASIC integrated circuits.”)
Zhou and Dulski together partly meet the limitations of claim 28. Dulski is directed to a similar field of art (a two-stage infrared image enhancement and object detection algorithm). Therefore, Zhou and Dulski are combinable. Zhou relies on a GPU to implement their proposed technique. Dulski indicates that an ASIC is an alternative suitable for complex algorithms. Modifying the system and method of Zhou by simple substitution of the GPU of Zhou for the ASIC indicated by Dulski would yield the expected and predictable result of complex, real-time processing. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine Zhou and Dulski in this way.
The combination of Zhou and Dulski does not disclose the following; however, Conti discloses:
process a first artificial neural network (ANN) (See the network in Fig. 1(b).)
wherein the first ANN is configured to perform deblur and denoise functions to an image (See the Abstract: “The structure of the model is based on two sub models devoted, respectively, to deblurring and denoising of an input image.”), and
Zhou, Dulski, and Conti together disclose the limitations of claim 28. Conti is directed to a similar field of art (a convolutional neural network dedicated to deblurring and denoising of images). Therefore, Zhou, Dulski, and Conti are combinable. As currently recited, claim 28 is interpreted to describe a branching approach in which an image is independently processed in multiple ways. Modifying the system and method of Zhou and Dulski by adding a separate/independent function of jointly deblurring and denoising the image of Zhou would yield the expected and predictable result of a system with diverse capabilities (i.e., an option for standalone deblurring/denoising and an two-stage option for contrast enhancement followed by object detection). Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine Zhou, Dulski, and Conti in this way.
Claim 29 is met by the combination of Zhou, Dulski, and Conti, wherein
The combination of Zhou, Dulski, and Conti discloses:
The apparatus of claim 21, wherein
And Conti further discloses:
the image is image processed by the first ANN in order to increase a rate of object detection performed by the second ANN (Note: the underlined limitation does not have patentable weight since it is an intended use limitation that merely states an outcome. The remainder of the claim (“the image is image processed by the first ANN”) is disclosed by the network depicted in Fig. 1(b).).
See the motivation to combine in the treatment of claim 28.
Claim 30 is met by the combination of Zhou, Dulski, and Conti, wherein
The combination of Zhou, Dulski, and Conti discloses:
The apparatus of claim 21, wherein
And Zhou further discloses:
weight parameters of the first ANN are trained to increase a rate of object detection for the second ANN (Note: the underlined limitation does not have patentable weight since it is an intended use limitation that merely states an outcome. The remainder of the claim is disclosed by the network parameters in Fig. 2.).
Claim 31 is met by the combination of Zhou, Dulski, and Conti, wherein
The combination of Zhou, Dulski, and Conti discloses:
The apparatus of claim 21, wherein
And Zhou further discloses:
weight parameters of the second ANN are trained to perform the object detection function (See network parameters implied in section 2.3 on pages 448-449.).
Claim 32 is met by the combination of Zhou, Dulski, and Conti, wherein
The combination of Zhou, Dulski, and Conti discloses:
The apparatus of claim 21, wherein
And Zhou further discloses:
The apparatus of claim 21, wherein the first ANN is configured to adjust image parameters of the image captured by the image sensor (See page 445: “Based on the superpixel segmentation, a multi-level unit-linking PCNN performs weighting to the image and highlight image foreground.”).
Claim 33 is met by the combination of Zhou, Dulski, and Conti, wherein
The combination of Zhou, Dulski, and Conti discloses:
The apparatus of claim 21, wherein
And Zhou further discloses:
the first ANN is further configured to perform at least one of wide dynamic range (WDR), high dynamic range (HDR), color tone mapping, and demosaicing functions (See page 445: “The image enhancement network uses contrast and structural features to enhance the input image.” Contrast adjustment paired with the use of structural features meets the claimed WDR and HDR.).
Claim 34 is met by the combination of Zhou, Dulski, and Conti, wherein
The combination of Zhou, Dulski, and Conti discloses:
The apparatus of claim 21, wherein
And Zhou further discloses:
the second ANN is configured to infer a type and location of the at least one object included in the image (See the refined saliency map in Fig. 3(h), which indicates the pixel locations of the object and the type of object with multilevel segmentation (see page 447).).
Claim 35 is met by the combination of Zhou, Dulski, and Conti for the reasons given in the treatment of claim 28.
Claim 37 is met by the combination of Zhou, Dulski, and Conti for the reasons given in the treatment of claim 23.
Claim 38 is met by the combination of Zhou, Dulski, and Conti for the reasons given in the treatment of claim 27.
Claim 39 is met by the combination of Zhou, Dulski, and Conti for the reasons given in the treatment of claim 33.
Claim 40 is met by the combination of Zhou, Dulski, and Conti, wherein
The combination of Zhou, Dulski, and Conti discloses:
The apparatus of claim 35, wherein
And Zhou further discloses:
the processor is further configured to perform at least two among brightness operation processing, contrast operation processing, defog operation processing, auto white balance operation processing, back light compensation operation processing, and decompression operation processing (See page 445: “The image enhancement network uses contrast and structural features to enhance the input image.” Contrast adjustment meets the claimed contrast operation processing.).
Claim(s) 36 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhou (Deep Neural Network Based Salient Object Detection with Image Enhancement, 17 November 2018, LNTCS, Vol. 11304, Pages 444-453), in view of Dulski (Evaluation of hardware implementation of the infrared image enhancement algorithm, 2012, QIRT, pages 1-9), in view of Conti (A Regularized Deep Learning Approach for Image De-Blurring, 2017, IML ’17, Pages 1-5), in view of Sharma et al. (Classification-Driven Dynamic Image Enhancement, 2018, CVF, Pages 4033-4041), hereinafter “Sharma”.
Claim 36 is met by the combination of Zhou, Dulski, Conti, and Sharma, wherein
The combination of Zhou, Dulski, and Conti discloses:
The apparatus of claim 35, wherein
The combination of Zhou, Dulski, and Conti does not disclose the following; however, Sharma discloses:
if the image is compressed, the image preprocessing ANN is configured to compensate for a compression loss of the image (See page 4033: "Examples include enhancement to remove the effects of blur, noise, poor contrast, and compression artifacts - or to boost image details.").
Motivation to combine:
Zhou, Dulski, Conti, and Sharma together teach the limitations of claim 36. Sharma is directed to a similar field of art (two-stage architecture for image classification). Therefore, Zhou, Dulski, Conti, and Sharma are combinable. Modifying the system and method of Zhou, Dulski, & Conti by adding the capability to "compensate for a compression loss of the image ", as taught by Sharma, would yield the expected and predictable result of application to a wider range of input images having degradations. Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to combine Zhou, Dulski, Conti, and Sharma in this way.
Allowable Subject Matter
Claims 21-27 would be allowable if rewritten or amended to overcome the double patenting rejection(s) set forth in this Office action.
Claim 21 recites: “an object detection module configured to process a second ANN configured to perform object detection function to an image output from the preprocessing module to detect at least one object included in the image output from the preprocessing module.” The underlined limitation makes clear that the output of the first ANN (deblurred and denoised) is passed to the second ANN. None of the prior art of record discloses or suggest this limitation.
Conti, as relied on above, does disclose a model with two sub-models devoted to deblurring and denoising an input image. However, Conti does not disclose a follow-up step of object detection on that deblurred and denoised image. In addition, the examiner believes that there is no teaching, suggestion, or motivation in the art to combine the two-stage approach of Zhou with the joint deblurring/denoising of Conti.
Claims 22-27 are dependent on claim 21 and are indicated as having the same allowable subject matter.
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONATHAN S LEE whose telephone number is (571)272-1981. The examiner can normally be reached 11:30 AM - 7:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571)270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Jonathan S Lee/Primary Examiner, Art Unit 2677
