Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 10/21/2024 is being considered by the examiner.
Drawings
The drawings are objected to because in Fig. 2, steps S102 and S107 lack labels on arrows pointing away from the steps. Applicant is advised to amend the figure to include labels of “YES” or “NO” on arrows pointing away from steps S102 and S107. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The disclosure is objected to because of the following informalities:
Paragraphs [0007], [0012]-[0014], [0030], [0036], [0041] recite “imagining unit 11,” which is not consistent with Drawing Figure 1 “imaging unit”
Paragraph [0019], line 6 recites “potions”; Applicant is advised to amend the term to “portions”
Appropriate correction is required.
Claim Objections
Claim 4 is objected to because of the following informalities: Claim 4 recites “imagining unit,” which as stated above, is not consistent with Drawing Figure 1. Applicant is advised to amend the term to “imagining unit,” or otherwise make amendments such that the term is consistent across the Drawings, Specification, and Claims. Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: “recognition unit” and “determination unit” in Claim 1, “rendering unit” in Claim 3, and “imagining unit” and “autoexposure unit” in Claim 4. The elements are equivalent to a “unit for recognition,” a “unit for determination,” a “unit for imagining,” and a “unit for autoexposure.”
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1 and 4 are rejected under 35 U.S.C. 103 as being unpatentable over Lin (US 2020/0327648 A1), in view of Bossard et al. (US 2022/0392209 A1).
Regarding Claim 1, Lin teaches “An image processing device” (Lin, [0160] discloses “The computer-readable instructions can be executed by the processor 220 to implement the method for image processing in the implementations. The processor 220 is configured to provide computing and control capabilities to support operation of the entire electronic device 200”; where a processor is an image processing device) “comprising:
a recognition unit configured to recognize (Lin, [0136] discloses “The detecting unit 27 is configured to determine, according to an image semantic recognition algorithm, whether the original image contains a target region”; where a detecting unit is a recognition unit); “and
a determination unit configured to determine to perform high-dynamic-range rendering when theLin, [0121] discloses “The synthesizing module 40 is configured to generate a target image by performing HDR synthesis on the multiple denoised images”; where HDR is high dynamic range; where a synthesizing module is a determination unit; see Fig. 15; where synthesizing module is executed after detecting unit in Fig. 15, and thus HDR synthesis, or high-dynamic-range rendering, occurs when a target region is detected, or an image has been recognized).
PNG
media_image1.png
351
436
media_image1.png
Greyscale
Fig. 1 of Lin
PNG
media_image2.png
581
356
media_image2.png
Greyscale
Fig. 15 of Lin
Although Lin, [0101] discloses “The target region can be a region where graphics of a user-defined type located, such as face region, famous building region, etc. The target region may also be set by the system according to big data analysis,” Lin does not explicitly teach “a recognition unit configured to recognize a flower image included in an image” (emphasis added).
However, in an analogous field of endeavor, Bossard teaches “a recognition unit configured to recognize a flower image included in an image” (Bossard, [0035] discloses “For example, one machine learning model may be trained to classify or otherwise recognize fruits in images, another machine learning model may be trained to classify or otherwise recognize flowers in images, and so forth. In one or more implementations, the machine learning models 208 may include a universal machine learning model that is trained to classify or otherwise recognize any type of object detected in an image”; where recognizing flowers in images is recognizing a flower image included in an image).
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified Lin to incorporate the teachings of Bossard by using a machine learning model to recognize flowers in images. The prior art Lin contained a device which differed from the claimed device by the substitution of some components with other components; Lin teaches an image processing unit including an image semantic recognition unit and a synthesizing unit that performs HDR image synthesis. Lin discloses “The target region can be a region where graphics of a user-defined type located, such as face region, famous building region, etc. The target region may also be set by the system according to big data analysis.” The claimed invention recites an image processing device comprising a recognition unit and HDR rendering on recognized flower images. The substituted component and its function (recognition unit configured to recognize a flower image) were known in the art; Bossard teaches a machine learning model trained to classify flowers in images. One of ordinary skill in the art could have substituted one known element (face region or famous building region) for another (flowers in images), and the results of the substitution would have been predictable. That is, it would have been predictable that substituting the user-defined type of face or famous building for flower images, and then processing the images as taught by Lin, would have resulted in the predictable results of flower images processed by HDR synthesis. Finally, one of ordinary skill in the art would be motivated to combine the Lin and Bossard references in order to improve image quality on images with brighter and darker regions: Lin, [0074] discloses “Therefore, in HDR synthesis, if weights of the brighter regions and darker regions in the figure in the middle as well as the weight of the very dark regions in the right-hand figure can be increased, proper exposure can be achieved in respective regions of the image and image quality can be improved.” Accordingly, the combination of Lin and Bossard discloses the invention of Claim 1.
Regarding Claim 4, the combination of Lin and Bossard teaches “An imaging apparatus comprising:
the image processing device according to claim 1” (see Lin and Bossard, above);
“an imagining unit configured to perform imaging” (Lin, [0117] discloses “As illustrated in FIG. 11, the apparatus for image processing includes a photographing module 10, a generating module 20, a denoising module 30, and a synthesizing module 40”; where photographing module is an imaging unit); “and
an autoexposure unit configured to regulate an exposure in the imaging to cause the imagining unit to capture a plurality of images that are to undergo the high-dynamic-range rendering” (Lin, [0157] discloses “The ISP processor is configured to control the image sensor to obtain the original image, and to generate the target image by performing the HDR synthesis on the multiple denoised images, which correspond to the original image and at least one reference image.” Lin, [0169] discloses “The statistical data determined by the ISP processor 91 can be sent to a control logic unit 92. For example, the statistical data may include statistical information of the image sensor 934 such as auto exposure, auto white balance, auto focus, flicker detection, black level compensation, and shading correction for lens 932…The ISP control parameters may include gain levels and color correction matrices for automatic white balance and color adjustment (e.g., during RGB processing), and shading correction parameters for lens 932”).
Claims 2 and 3 are rejected under 35 U.S.C. 103 as being unpatentable over Lin (US 2020/0327648), in view of Bossard et al. (US 2022/0392209 A1), further in view of Einecke et al. (US 2015/0138339 A1).
Regarding Claim 2, the combination of Lin and Bossard teaches “The image processing device according to claim 1, wherein
the determination unit determines to perform the high-dynamic-range rendering when having determined not to perform the high-dynamic-range rendering in the primary determination, and when the flower image has been recognized by the recognition unit” (Lin, [0121] discloses “The synthesizing module 40 is configured to generate a target image by performing HDR synthesis on the multiple denoised images”; where HDR is high dynamic range; where a synthesizing module is a determination unit; see Fig. 15).
The combination of Lin and Bossard does not explicitly teach “the determination unit performs a primary determination to determine whether to perform the high-dynamic-range rendering in accordance with a contrast of the image.”
However, in an analogous field of endeavor, Einecke teaches “the determination unit performs a primary determination to determine whether to perform the high-dynamic-range rendering in accordance with a contrast of the image” (Einecke, [0037] discloses “If the contrast is too strong, meaning that underexposed and/or overexposed areas are present to such an extent that no meaningful further image processing is possible, it is decided that the control system is switched to an HDR mode in order to ensure an image quality suitable for further processing of the image. Therefore in step S3 the intensity range determined is compared to a threshold in step 53”).
It would have been obvious to one of ordinary skill in the art before the effective filing
date of the claimed invention to have modified the combination of Lin and Bossard to incorporate the teachings of Einecke by determining a contrast of captured images and switching to an HDR mode depending on the contrast exceeding a threshold. One of ordinary skill in the art would be motivated to combine the Lin, Bossard, and Einecke references in order to use HDR only when necessary: Einecke, [0017] discloses “Thus, the additional data processing for generating an HDR image is necessary only in cases where the lighting conditions are in fact difficult. All other situations, where lighting conditions do not lead to a high contrast between different areas of the image, single images captured by the camera can be used. Images having a high contrast in the sense of the inventions are images that have areas that area overexposed and/or areas that are underexposed to such an extent that image processing as desired, e.g. for obstacle avoidance cannot be performed satisfyingly.” Accordingly, the combination of Lin, Bossard, and Einecke discloses the invention of Claim 2.
Regarding Claim 3, the combination of Lin, Bossard, and Einecke teaches “The image processing device according to claim 2, comprising
a rendering unit configured to subject a plurality of first images having a first exposure difference to the high-dynamic-range rendering when the determination unit has determined to perform the high-dynamic-range rendering in the primary determination” (Lin, [0054] discloses “Further, the initial exposure can be preset to three levels, that is, a first exposure, a second exposure, and a third exposure. The first exposure is smaller than the second exposure. The second exposure is smaller than the third exposure. For example, the first exposure is EV−, the second exposure is EV0, and the third exposure is EV+”; where a second exposure is a first exposure difference. Lin, [0104] discloses “At block 402, when the original image contains the target region, a first exposure compensation amount is determined according to pixel brightness of the target region.” Lin, [0106] discloses “At block 404, a corresponding reference image is generated by adjusting brightness of the target region in the original image according to the first exposure compensation amount”; where reference images generated at block 404 are denoised and used to perform HDR synthesis (see Fig. 1)), “and to subject a plurality of second images having a second exposure difference to the high-dynamic-range rendering when the determination unit has determined to perform the high-dynamic-range rendering in response to recognition of the flower image by the recognition unit, the second exposure difference being smaller than the first exposure difference” (Lin, [0054] discloses “Further, the initial exposure can be preset to three levels, that is, a first exposure, a second exposure, and a third exposure. The first exposure is smaller than the second exposure. The second exposure is smaller than the third exposure. For example, the first exposure is EV−, the second exposure is EV0, and the third exposure is EV+”; where a first exposure is a second exposure difference, a first exposure being smaller than the second exposure. Lin, [0105] discloses “At block 403, a second exposure compensation amount is determined according to pixel brightness of a non-target region.” Lin, [0107] discloses “At block 405, another corresponding reference image is generated by adjusting brightness of the non-target region in the original image according to the second exposure compensation amount”; where reference images generated at block 405 are denoised and used to perform HDR synthesis (see Fig. 1)).
PNG
media_image3.png
545
454
media_image3.png
Greyscale
Fig. 9 of Lin
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Tanaka et al. (US 2022/0239825 A1) discloses a method of performing high dynamic range rendering on a partial region of interest segmented from a captured image, including varying exposure times/exposure patterns across images.
Inaba et al. (US 2019/0147256 A1) discloses a method of identifying objects in vehicle surroundings, including flowers; see [0039].
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CAROLINE TABANCAY DUFFY whose telephone number is (703)756-1859. The examiner can normally be reached Monday - Friday 8:00 am - 5:30 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached at 5712723382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/CAROLINE TABANCAY DUFFY/Examiner, Art Unit 2662
/AMANDEEP SAINI/Supervisory Patent Examiner, Art Unit 2662
Prosecution Insights