Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 1-3 are objected to because of the following informalities:
Claim 1 recites in line 3 “than predetermined brightness”, should read “than a predetermined brightness”.
Claim 1 recites in line 4 “into first brightness”, should read “into a first brightness”.
Claim 1 recites in lines 6-7 “the extraction data converted is reflected”, should read “the converted extraction data is reflected”.
Claim 2 recites in line 2 “a conversion model machine-learned”, should read “a machine-learned conversion model”. Claim 3 recites similar issues.
Claim 2 recites in line 4 “the extraction data input into”, should read “the extraction data
Claim 3 recites in line 3 “to convert an image in the first region”, should read “to convert a first region in the image”.
Claim 3 recites “an input first/second conversion condition”, should read “a
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) are:
Claim 1 recites “an extractor configured to” in line 2. The corresponding structure in the specification is the recited program at Para. 46together with the general-purpose computer running the specific program disclosed in Paras. 47-49.
Claim 1 recites “a convertor configured to” in line 4. The corresponding structure in the specification is the recited program at Para. 46. Accordingly, the corresponding structure is a general-purpose computer running the specific program disclosed in Paras. 50-52.
Claim 1 recites “an output unit configured to” in line 6. The corresponding structure in the specification is the recited program and display at Paras. 45-46. Accordingly, the corresponding structure is a general-purpose computer running the specific program disclosed in Para. 53 to control a display.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because:
Claim 10 recites a program per se but does not have a physical or tangible form as no accompanying structure is recited. See MPEP 2106.03.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 3 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 3, “the learning image data” is recited in lines 4 and 8. Claim 2 defines “a plurality of pieces of learning image data”, leaving it unclear whether “the learning image data” of claim 3 refers to one specific element or a plurality of pieces of learning image data.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1 and 9-10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Okuda et al (JP Patent Pub. No. 2009035162 A, machine translation attached).
Regarding claim 1, Okuda teaches an information processing apparatus, comprising: an extractor configured to extract a first region having brightness equal to or greater than predetermined brightness in an image shown by image data imaged by a moving body (Pg. 4, “The image processing unit 21 detects the luminance of the points constituting the image included in the image data input from the camera 10 (step S2). Here, the points constituting the image represent pixels or a certain area… Further, following step S2, the image processing unit 21 performs image processing on a portion having a luminance value equal to or higher than the luminance value”); a converter configured to convert extraction data of the first region extracted into first brightness lower than the predetermined brightness (Pg. 4, “Specifically, the image processing unit 21 reduces the lightness in a portion having a predetermined luminance value or more.”); and an output unit configured to output the image data in which the extraction data converted is reflected (Pg. 4, “The image processing unit 21 outputs the image subjected to the image processing as a rear view image, and outputs the rear view image as image data to the image display unit 31.”).
Claim 9 and 10 both correspond to claim 1 and are rejected under the same analysis.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 4, and 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Oduka in view of Gordon et al. (WIPO Pub. No. 2020/095233 A1, pdf attached).
Regarding claim 2, Oduka teaches all of the elements of claim 1, as stated above, as well as wherein the converter converts the extraction data input into the first brightness (Pg. 4, “Specifically, the image processing unit 21 reduces the lightness in a portion having a predetermined luminance value or more.”).
They do not explicitly disclose using a conversion model machine-learned to convert a plurality of pieces of learning image data having an imaging condition different from that of the extraction data and having a feature of the extraction data.
Gordon teaches wherein the converter converts, by using a conversion model machine-learned to convert a plurality of pieces of learning image data having an imaging condition different from that of the extraction data and having a feature of the extraction data, the extraction data input into the first brightness (Pare. 26, “In a second optional step, the image can be segmented, with regions around a bright spot readied for further processing.”; Para. 47, “Images 402 are original images. Images 404 have one or more synthetic bright spots added. As is apparent, various types of bright spot size and radial features are shown. Images 406 are corrected by neural network denoising process”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Oduka to incorporate the teachings of Gordon to include a machine-learned model trained on learning image data. Oduka discloses a method of reducing brightness in images taken from a vehicle to avoid “dazzling” the viewer of the image, however they do not disclose utilizing a neural network to perform this method. Gordon discloses a neural network system used to reduce bright spots in images. One of ordinary skill in the art would have recognized that implementing the neural network of Gordon into the method of Oduka would have been a predictable improvement using known techniques. Gordon further discloses that “bright spot removal can improve a vehicle’s imaging system scene classification and object detection”, Para. 44, providing a clear motivation of improvement.
Regarding claim 4, Oduka as modified in view of Gordon teaches all of the elements of claim 2, as stated above, as well as wherein the image data imaged is image data imaged by an imaging device that is provided in a moving body periodically moving on a predetermined route and can image an outside of the moving body (Pg. 4, “As shown in FIG. 4, the camera 10 of the mounted vehicle 1 always captures the rear view of the mounted vehicle 1 (step S1).”), and the learning image data is data based on the image data imaged on the predetermined route (Para. 23, “Modern learning based algorithms work exceptionally well for those data distribution sets for which they have been trained on. When machine learning algorithms are presented with data outside this distribution, or when using adversarial examples, accuracy, speed, and other performance measures of these algorithms can suffer.”; Para. 52, “Training neural networks in a supervised or semi-supervised way requires high quality training data.”, Although they do not explicitly state that the learning image data is data based on the image data imaged on the predetermined route, given that both Oduka and Gordon process images in relation to vehicle imaging systems, and Gordon discloses that proper, high quality training data is extremely important for neural network efficiency, it would have been obvious to one of ordinary skill in the art to train on images specifically from the predetermined route.).
Regarding claim 5, Oduka as modified in view of Gordon teaches all of the elements of claim 4, as stated above, as well as wherein the imaging condition comprises a time zone (Pg. 5, “In the above embodiment, the mounted vehicle 1 has been described when traveling at night. However, as a light source that the driver feels dazzling in the rear view of the other mounted vehicle 1, driving can be performed even when sunlight is present during daytime traveling.”; Para. 3, “Daytime glares can often be attributed to reflections off mirror or glass surfaces that reduce details in the vicinity of the reflective object. Night time photography is particularly susceptible to glare around streetlights or other point sources, and even portrait photography can be affected by eyeglass or clothing reflections.”, Both Oduka and Gordon acknowledge that the time of day can affect the processing of the method, and cameras routinely store timestamp information. One of ordinary skill in the art would have recognized that associating a time of day with a time zone allows for the correct interpretation and application of adjustments at different times).
Regarding claim 6, Oduka as modified in view of Gordon teaches all of the elements of claim 2, as stated above, as well as wherein the image data and the learning image data comprise, as additional data, at least one selected from the group consisting of information of a position at which the image data and the learning image data are acquired, direction information of a direction in which the image data and the learning image data are imaged (Pg. 6, “At this time, the direction of the camera 10 is changed by the drive device provided in the camera 10 based on the output from the detection device that detects the steering angle of the steering wheel.”), time information of time at which the image data and the learning image data are imaged (See analysis of claim 5), weather information of weather in imaging the image data and the learning image data (Para. 33, “Advantageously, such neural network processed image normalization can, for example, reduce computer vision algorithm failure in high noise environments, enabling these algorithms to work in environments where they would typically fail due to noise related reduction in feature confidence. Typically, this can include but is not limited to low light environments, foggy, dusty, or hazy environments, or environments subject to light flashing or light glare.”, Gordon discloses that noisy environments or conditions cause typical image processing algorithms to fail, with weather conditions such as fog being given as an example. One of ordinary skill in the art would have been motivated to incorporate weather information as part of the imaging condition to mitigate these known effects, particularly in vehicular based imaging systems which are constantly outside), and feature information by which a feature of a subject image of the image data and the learning image data can be identified (Para. 38, disclose a convolutional neural network architecture which performs feature extraction).
Claim(s) 3 and 8 as best understood are rejected under 35 U.S.C. 103 as being unpatentable over Oduka as modified in view of Gordon, further in view of Murata et al. (US Patent Pub. No. 2020/0285884 A1).
Regarding claim 3, Oduka as modified in view of Gordon teaches all of the elements of claim 2, as stated above, as well as wherein the conversion model comprises a first conversion model machine-learned to convert an image in the first region shown by the learning image data into the first brightness (See analysis of claim 2) and the converter converts the extraction data satisfying an input first conversion condition by using the first conversion model (See analysis of claim 1).
They do not explicitly disclose wherein a second conversion model machine-learned to convert a region of the first region in which an imaging object having the predetermined brightness or more does not exist into the first brightness without converting a region in which the imaging object having the predetermined brightness or more exists in the image shown by the learning image data into the first brightness, and converts the extraction data satisfying an input second conversion condition by using the second conversion model.
Murata teaches wherein a second conversion model machine-learned to convert a region of the first region in which an imaging object having the predetermined brightness or more does not exist into the first brightness without converting a region in which the imaging object having the predetermined brightness or more exists in the image shown by the learning image data into the first brightness, and converts the extraction data satisfying an input second conversion condition by using the second conversion model (Para. 93, “In the above-described working example, the case of deletion of the neon signboard 312 by the display of the virtual image for use in the masking has been described. However, when the neon signboard is necessary for arrival to a destination, it is not desirable to perform the masking process on this neon signboard 312. Accordingly, in a relation between the region having the high luminance in the image captured by the image capturing unit 11 and the driving position, when this region having the high luminance corresponds to the destination, cancellation of the masking process is considered.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Oduka and Gordon to incorporate the teachings of Murata to include wherein a second conversion model machine-learned to convert a region of the first region in which an imaging object having the predetermined brightness or more does not exist into the first brightness without converting a region in which the imaging object having the predetermined brightness or more exists in the image shown by the learning image data into the first brightness, and converts the extraction data satisfying an input second conversion condition by using the second conversion model. Both Oduka and Gordon disclose methods for reducing brightness in images taken by a vehicular imaging system. Murata discloses a system to mask high luminance regions from images taken in a car, where the region is not masked if it is relevant to the destination. One of ordinary skill in the art would have understood that implementing the context aware processing model of Murata into the modified system of Oduka and Gordon would have predictably improved the handling of bright spots in an image, avoiding the degradation of safety-critical lights such as taillights and traffic signals while still reducing brightness in other non-critical regions.
Regarding claim 8, Oduka teaches all of the elements of claim 1, as stated above, and when modified in view of Gordon and Murata also teach wherein the converter decreases brightness of a streetlight shown in the image (See analysis of claim 1) and suppresses a decrease in the brightness of the streetlight compared to a decrease in brightness of at least one selected from the group consisting of a traffic light, a lamp of a car, a light at a construction site, and a light emitter worn by a person shown in the image (Pg. 5, “For this reason, the entire image is not darkened according to the luminance of the light part, and the rear view can be visually recognized while preventing the dazzling of the light of the succeeding vehicle 2”; Para. 27, “Multiple bright spots can be identified in the image and sequentially processed for bright spot mitigation or removal.”; Para. 41, “Conventional photos can be improved, or certain areas with bright spots can be selected for improvement, with other bright spot features left for aesthetic purposes.”; Murata; Para. 93, “However, when the neon signboard is necessary for arrival to a destination, it is not desirable to perform the masking process on this neon signboard 312.”, Gordon discloses processing multiple bright spots in an image, and alongside Oduka teaches the capability to improve certain areas with others left untouched. Murata discloses a context aware masking process for bright objects. One of ordinary skill in the art would have recognized that decreasing the brightness of one specific object in an image (such as a safety-critical light) less than the decrease in brightness of other bright objects in the image is a predictable variation of the disclosed multiple bright spots and area specific processing of Gordon and Oduka in view of the context aware brightness processing of Murata).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Oduka in view of Rosebrock (NPL, “OpenCV Gamma Correction”, published 2015, relevant pages attached)
Regarding claim 7, Oduka teaches all of the elements of claim 1, as stated above. They do not explicitly disclose wherein the extractor compares the image data and conversion data obtained by performing gamma correction processing on the image data to specify the first region of the image and extracts the extraction data of the first region from the image data. However, they do perform luminance detection on images for extraction.
Rosebrock teaches wherein performing gamma correction processing on the image data allows for a simple comparison of brightness regions (Fig. 7, shows two pictures of the same mountain range with two separate gamma values applied to it, clearly highlighting the difference in brightness throughout).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified Oduka to incorporate the teachings of Rosebrock to include wherein the extractor compares the image data and conversion data obtained by performing gamma correction processing on the image data to specify the first region of the image and extracts the extraction data of the first region from the image data. Oduka discloses performing luminance detection on images, however they do not explicitly disclose how this detection is performed. Rosebrock discloses that gamma correction is a well-known method for adjusting the brightness of images, with multiple examples showing that increasing the gamma allows for easier detection of darker regions and highlights brighter regions when compared to a lower gamma version of the same image. One of ordinary skill in the art would have understood that implementing the gamma correction processing of Rosebrock into the method of Oduka would predictably improve the detection of bright spots by highlighting differences in brightness between the images.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID A WAMBST whose telephone number is (703)756-1750. The examiner can normally be reached M-F 9-6:30 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571)272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DAVID ALEXANDER WAMBST/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights