DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged that application claims priority to foreign application with application number JP 2023-142160 dated 1 September 2023. Copies of certified papers required by 37 CFR 1.55 have been received. Priority is acknowledged under 35 USC 119(e) and 37 CFR 1.78.
Information Disclosure Statement
The IDS dated 12 August 2024 has been considered and placed in the application file.
Specification - Title
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The following title is suggested: Correcting lens distortion using frequency.
Claim Interpretation
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claims 4, 6, 9 and 13 recite “or.” Since “or” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f), is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f), is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f), because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
acquisition processing to acquire in claims 1 and 10;
correction processing to correct in claims 1 and 10; and
generation processing to generate in claims 1 and 10.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 5-6, 8-12, 14-15 and 17 (all claims except 4, 7, 13 and 16) are rejected under 35 U.S.C. 102(a)(1) and/or (a)(2) as being anticipated by US Patent Publication 2020 0007734 A1, (Kagawa et al.). References are listed in the Notice of Cited References when they were first cited. If a reference is not identifiable (e.g., due to a typo), it can be identified by searching for the quoted text.
[AltContent: textbox (Kagawa et al., assigned to Canon, Fig. 7, showing frequency analysis.)]
PNG
media_image1.png
735
520
media_image1.png
Greyscale
Claim 1
Regarding Claim 1, Kagawa et al. disclose an image processing apparatus ("image processing apparatus," paragraph [0005]) comprising:
a processor ("at least one processor," paragraph [0005]); and
a memory storing a program which, when executed by the processor, causes the image processing apparatus ("at least one memory coupled to the at least one processor, wherein the at least one memory stores an instruction that causes, when executed by the at least one processor, the image processing apparatus to," paragraph [0005]) to
perform acquisition processing to acquire a frequency of each pixel value from an input image ("The frequency separation unit 607 specifies the spatial frequency of the brightness image, and separates the image into a high-frequency component and a low-frequency component based on the specified spatial frequency," paragraph [0051]),
perform correction processing to correct the frequency acquired in the acquisition processing, based on distortion of the input image ("The contrast correction unit 609 executes contrast correction for the input high-frequency component," paragraph [0051]), and
perform generation processing to generate an assist image which indicates the frequency of each pixel value, based on the frequency of the correction processing ("The brightness/color difference combining unit 611 combines the brightness image data after the processing and the color difference image data, thereby generating output image data," paragraph [0052]).
Claim 2
Regarding Claim 2, Kagawa et al. disclose the image processing apparatus according to claim 1, wherein in the correction processing, the frequency of each pixel value is increased in a region of which the distortion is determined to be large ("The contrast correction unit 609 performs contrast correction processing for the high-frequency component image obtained in step S706 (step S708)," paragraph [0062] where contrast correction includes increasing the frequency of a pixel value).
Claim 3
Regarding Claim 3, Kagawa et al. disclose the image processing apparatus according to claim 1, wherein in the correction processing, the frequency of each pixel value is decreased in a region of which the distortion is determined to be small ("The contrast correction unit 609 performs contrast correction processing for the high-frequency component image obtained in step S706 (step S708)," paragraph [0062] where contrast correction includes decreasing the frequency of a pixel value).
Claim 5
Regarding Claim 5, Kagawa et al. disclose the image processing apparatus according to claim 1, wherein in the correction processing, for each of a plurality of regions of the input image, the frequency of each pixel value in the region is corrected, based on distortion of an image in the region ("FIGS. 3A and 3B show examples of the D range compression curves applied to the areas 102 and 103, respectively. Referring to FIGS. 3A and 3B, the abscissa represents an input brightness range, and the ordinate represents an output brightness range," paragraph [0033] where areas are regions and range compression is correcting the frequency of pixel values).
Claim 6
Regarding Claim 6, Kagawa et al. disclose the image processing apparatus according to claim 5, wherein the plurality of regions are a plurality of regions arranged in a grid pattern or a plurality of regions arranged concentrically ("FIG. 1 shows an example in which an image 101 is divided into 4x3 areas, and FIGS. 2A and 2B show examples of histograms representing brightness distributions in areas 102 and 103 shown in FIG. 1, respectively," paragraph [0033] where a 4 X 3 area is a grid).
Claim 8
Regarding Claim 8, Kagawa et al. disclose the image processing apparatus according to claim 1, wherein in the generation processing, the assist image is generated regarding the frequency of each pixel value in an invalid region of the input image as 0 ("In this processing, the input high-frequency component image is multiplied by a coefficient k. In a case in which an expression close to the scene at the time of image capturing is requested, k=l±Delt. (Delt. is 0 or a sufficiently small predetermined value) is set," paragraph [0062]).
Claim 9
Regarding Claim 9, Kagawa et al. disclose the image processing apparatus according to claim 1, wherein the assist image is a waveform monitor image which indicates distribution and frequency of a brightness value ("In addition, the dynamic range obtaining unit 605 obtains the brightness data of the bright and dark portions on the output side (step S704). FIG. 9B shows a table of brightness values for each sheet to output (print) an image," paragraph [0058]), a histogram image which indicates distribution and frequency of a brightness value ("For such a histogram, a cumulative curve that accumulates the number of pixels in the brightness order is defined as a D range compression curve," paragraph [0070]), a vector scope image which indicates distribution and frequency of a combination of hue and chroma, or a chromaticity diagram image which indicates distribution and frequency of chromaticity.
Claim 10
Regarding Claim 10, Kagawa et al. disclose an image processing apparatus ("image processing apparatus," paragraph [0005])comprising:
a processor ("at least one processor," paragraph [0005]); and
a memory storing a program which, when executed by the processor, causes the image processing apparatus ("at least one memory coupled to the at least one processor, wherein the at least one memory stores an instruction that causes, when executed by the at least one processor, the image processing apparatus to," paragraph [0005]) to
perform detection processing to detect, from an input image, frequency components higher than a threshold ("The frequency separation unit 607 specifies the spatial frequency of the brightness image, and separates the image into a high-frequency component and a low-frequency component based on the specified spatial frequency," paragraph [0051]),
perform correction processing to correct the threshold based on distortion of the input image ("The contrast correction unit 609 executes contrast correction for the input high-frequency component," paragraph [0051]), and
perform image processing, to emphasize the high frequency components detected in the detection processing, on the input image ("The brightness/color difference combining unit 611 combines the brightness image data after the processing and the color difference image data, thereby generating output image data," paragraph [0052]).
Claim 11
Regarding Claim 11, Kagawa et al. disclose the image processing apparatus according to claim 10, wherein in the correction processing, the threshold is increased in a region of which the distortion is determined to be large ("The contrast correction unit 609 performs contrast correction processing for the high-frequency component image obtained in step S706 (step S708)," paragraph [0062] where contrast correction includes increasing the frequency of a pixel value).
Claim 12
Regarding Claim 12, Kagawa et al. disclose the image processing apparatus according to claim 10, wherein in the correction processing, the threshold is decreased in a region of which the distortion is determined to be small ("The contrast correction unit 609 performs contrast correction processing for the high-frequency component image obtained in step S706 (step S708)," paragraph [0062] where contrast correction includes decreasing the frequency of a pixel value).
Claim 14
Regarding Claim 14, Kagawa et al. disclose the image processing apparatus according to claim 10, wherein in the correction processing, for each of a plurality of regions of the input image, the threshold in the region is corrected based on distortion of an image in the region ("FIGS. 3A and 3B show examples of the D range compression curves applied to the areas 102 and 103, respectively. Referring to FIGS. 3A and 3B, the abscissa represents an input brightness range, and the ordinate represents an output brightness range," paragraph [0033] where areas are regions and range compression is correcting the frequency of pixel values).
Claim 15
Regarding Claim 15, Kagawa et al. disclose the image processing apparatus according to claim 14, wherein the plurality of regions are a plurality of regions arranged in a grid pattern, or a plurality of regions arranged concentrically ("FIG. 1 shows an example in which an image 101 is divided into 4x3 areas, and FIGS. 2A and 2B show examples of histograms representing brightness distributions in areas 102 and 103 shown in FIG. 1, respectively," paragraph [0033] where a 4 X 3 area is a grid).
Claim 17
Regarding Claim 17, Kagawa et al. disclose an image processing method ("image processing apparatus," paragraph [0005]) comprising:
acquiring a frequency of each pixel value from an input image ("The frequency separation unit 607 specifies the spatial frequency of the brightness image, and separates the image into a high-frequency component and a low-frequency component based on the specified spatial frequency," paragraph [0051]);
correcting the acquired frequency based on distortion of the input image ("The contrast correction unit 609 executes contrast correction for the input high-frequency component," paragraph [0051]); and
generating an assist image which indicates the frequency of each pixel value, based on the corrected frequency ("The brightness/color difference combining unit 611 combines the brightness image data after the processing and the color difference image data, thereby generating output image data," paragraph [0052]).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 4, 7, 13 and 16 (all remaining claims) are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2020 0007734 A1, (Kagawa et al.) in view of US Patent Publication 2015 0254872 A1, (de Almeida Barreto et al.). The references are listed in a PTO-892 from the Office Action in which they are first used. If a reference is not identifiable (e.g., due to a typo), it can be identified by searching for the quoted text.
Claim 4
Regarding Claim 4, Kagawa et al. teach the image processing apparatus according to claim 1, as noted above.
[AltContent: textbox (de Almeida Barreto et al. Fig. 5, showing calibration based on lens factors.)]
PNG
media_image2.png
552
676
media_image2.png
Greyscale
Kagawa et al. is not relied upon to explicitly teach all of lenses.
However, de Almeida Barreto et al. teach wherein the input image is an image captured using a fisheye lens, an equidistant cylindrical type image, or an equidistant projection type image ("fisheye lenses provide a wide field-of-view that proved to be beneficial for tasks like egomotion estimation [25] and visual place recognition [26]," paragraph [0004]).
Therefore, taking the teachings of Kagawa et al. and de Almeida Barreto et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify “Image Processing Apparatus” as taught by Kagawa et al. to use “Method for Aligning and Tracking Point Regions” as taught by de Almeida Barreto et al., showing that Kagawa et al. and de Almeida Barreto et al. are analogous art because both are image processing. The suggestion/motivation for combination is that, “The interest on feature tracking using image alignment techniques dates back to the 80s, when Lucas and Kanade [1] formulated the tracking using a brightness constancy assumption” as noted by the de Almeida Barreto et al. disclosure in paragraph [0009], which also motivates combination because the combination would predictably have a higher efficiency as there is a reasonable expectation that feature tracking is complex and will require adaption for several factors; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 7
Regarding claim 7, Kagawa et al. teach the image processing apparatus according to claim 1, as noted above.
Kagawa et al. is not relied upon to explicitly teach all of lens used for capturing the input image.
However, de Almeida Barreto et al. teach wherein the program, when executed by the processor, further causes the image processing apparatus to perform second acquisition processing to acquire information on a lens used for capturing the input image ("FIG. 5 compares the estimation of the focal length obtained from the uRD-KLT tracking information with the explicit SIC calibration results obtained for a certain number of zoom positions," paragraph [0085]),
wherein in the correction processing, the frequency acquired in the acquisition processing is corrected based on distortion of the input image determined based on the information ("It can be seen that, since the s is robustly estimated during the camera operation, it is possible to compute the focal length on-the-fly with a median error of 1.9% for the given example of 35 frames," paragraph [0085]).
Kagawa et al. and de Almeida Barreto et al. are combined as per claim 4.
Claim 13
Regarding claim 13, Kagawa et al. teach the image processing apparatus according to claim 10, as noted above.
Kagawa et al. is not relied upon to explicitly teach all of lenses.
However, de Almeida Barreto et al. teach wherein the input image is an image captured using a fisheye lens, an equidistant cylindrical type image, or an equidistant projection type image ("fisheye lenses provide a wide field-of-view that proved to be beneficial for tasks like egomotion estimation [25] and visual place recognition [26]," paragraph [0004]).
Kagawa et al. and de Almeida Barreto et al. are combined as per claim 4.
Claim 16
Regarding claim 16, Kagawa et al. teach the image processing apparatus according to claim 10, as noted above.
Kagawa et al. is not relied upon to explicitly teach all of lens used for capturing the input image.
However, de Almeida Barreto et al. teach wherein the program, when executed by the processor, further causes the image processing apparatus to perform acquisition processing to acquire information on a lens used for capturing the input image("FIG. 5 compares the estimation of the focal length obtained from the uRD-KLT tracking information with the explicit SIC calibration results obtained for a certain number of zoom positions," paragraph [0085]),
wherein in the correction processing, the threshold is corrected based on distortion of the input image determined based on the information ("It can be seen that, since the s is robustly estimated during the camera operation, it is possible to compute the focal length on-the-fly with a median error of 1.9% for the given example of 35 frames," paragraph [0085]).
Kagawa et al. and de Almeida Barreto et al. are combined as per claim 4.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US Patent Publication 2020 0296302 A1 to Shiro et al. discloses An image capturing device includes an imaging device and circuitry. The imaging device captures an image. The circuitry defines a point of interest in the image, converts the defined point of interest in accordance with attitude information of the image capturing device, and cuts out a viewable area from the image.
US Patent Publication 2020 0219202 A1 to Macciola et al. discloses capturing image data depicting a document; defining a plurality of candidate edge points within the image data; and defining four sides of a tetragon based on at least some of the plurality of candidate edge points; wherein each side of the tetragon corresponds to a different side of the document; wherein an area of the tetragon comprises at least a threshold percentage of a total area of the digital image; and wherein the tetragon bounds the digital representation of the document.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATH E WELLS whose telephone number is (703)756-4696. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mehmood can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Heath E. Wells/Examiner, Art Unit 2664
Date: 7 August 2026
Prosecution Insights