DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of claims 1-24 are pending below.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 9*10/2024 was filed and considered. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-24 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Sinha (US 2021/0158096).
Claim 1:
Sinha (US 2021/0158096) anticipated the following subject matter:
An image processing apparatus comprising: a format conversion section that converts RGB data to RAW data (abstract detail director convert RGB images into Raw Bayer format for image analytics; 0021 detail a trained generative model configured to receive an image in an RGB format and generate a corresponding image in a raw Bayer format).
Claim 2:
Sinha teaches:
The image processing apparatus according to claim 1, wherein the format conversion section is generated by adversarial training with a determination section that determines authenticity of the RAW data arising from conversion from the RGB data with respect to RAW data before being converted to the RGB data (0013 detail generative adversarial network (GAN) as the neural network; 0019 detail neural network for the generation (conversion) with consideration for error between ground truth to raw image (authenticity); figure 7 and 0065 detail further).
Claim 3:
The image processing apparatus according to claim 1, wherein the format conversion section downscales the RAW data arising from conversion after converting the RGB data to the RAW data (figure 18 and 0086 detail Raw2rec to raw Bayer with lower memory requirement (downscale)).
Claim 4:
The image processing apparatus according to claim 1, wherein the format conversion section converts learning data including the RGB data (0008 detail convert RGB into raw format and train generative model using the converted data from RGB to RAW; 0029) and a training recognition result to the learning data including the RAW data and the training recognition result (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077).
Claim 5:
The image processing apparatus according to claim 4, further comprising: a RAW data recognition section that is generated by learning using the learning data including the RAW data and the training recognition result and executes image recognition processing for an image of the RAW data (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 and paragraph 0069; figure 13 and 0077).
Claim 6:
The image processing apparatus according to claim 5, further comprising: an imaging apparatus that captures the image and outputs the image as the RGB data, wherein the format conversion section converts the RGB data output from the imaging apparatus, to the RAW data, and the RAW data recognition section executes the image recognition processing on a basis of the RAW data for which format conversion has been executed by the format conversion section (0012 detail image signal processing (ISP) circuitry configure to receive an unlabeled ground truth image in a raw Bayer format and generate an image in a RGB format; a generative model configured to receive the image in the RGB format and generate an estimated raw image in the raw Bayer format; and an error generation circuitry configured to receive the unlabeled ground truth image in the raw Bayer format and the estimated raw image and to generate an error between the unlabeled ground truth image and the estimated raw image; wherein the generative model is configured to train based on a back propagation of the error; above teaches identify patterns (training recognition) in paragraph 0051, figure 10 and 0069 and figure 13 and 0077).
Claim 7:
The image processing apparatus according to claim 6, wherein the imaging apparatus includes an imaging element that captures the image and outputs the image as the RAW data, and a signal processing section that executes demosaicing processing for the RAW data output from the imaging element, to convert the RAW data to the RGB data and output the RGB data (figure 1 and 0054).
Claim 8:
The image processing apparatus according to claim 5, further comprising: an imaging element that captures the image and outputs the image as the RAW data, wherein the RAW data recognition section executes the image recognition processing on a basis of the RAW data output from the imaging element (0006-0007 detail output raw format with neural network coupled to sensor for generate image analytics for image classification or object detection (image recognition)).
Claim 9:
The image processing apparatus according to claim 1, further comprising: a RAW data recognition section that is generated by retraining a trained RGB recognition section that executes image recognition processing for an image of the RGB data by using the RAW data for which format conversion from the RGB data has been executed by the format conversion section, and executes image recognition processing for an image of the RAW data (above teaches RGB to RAW to generate image analytic for training, where 0013 detail generative adversarial network (GAN), where generative adversarial network is retraining that explicitly incorporates adversarial examples into the training process to improve model robustness).
Claim 10:
The image processing apparatus according to claim 1, wherein the RAW data includes a Bayer format, a multi-spectrum format, a monochrome format, a polarization format, or a depth map format (0006 detail image data into raw Bayer format).
Claim 11:
An information processing method comprising a step of: converting RGB data to RAW data (abstract detail director convert RGB images into Raw Bayer format for image analytics; 0021 detail a trained generative model configured to receive an image in an RGB format and generate a corresponding image in a raw Bayer format).
[Claim 12] A program that causes a computer to function as: a format conversion section that converts RGB data to RAW data (abstract detail director convert RGB images into Raw Bayer format for image analytics; 0021 detail a trained generative model configured to receive an image in an RGB format and generate a corresponding image in a raw Bayer format).
[Claim 13] An image processing apparatus comprising: a RAW data recognition section that executes image recognition processing on a basis of an image of RAW data (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077).
[Claim 14] The image processing apparatus according to claim 13, wherein the RAW data recognition section is generated by learning based on learning data including the RAW data and a training recognition result, and the learning data including the RAW data and the training recognition result includes learning data arising from format conversion from learning data including RGB data and the training recognition result (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077).
[Claim 15] The image processing apparatus according to claim 13, wherein the RAW data recognition section arises from retraining a trained RGB recognition section that executes image recognition processing for an image of RGB data by using the RAW data generated by format conversion from the RGB data (above teaches RGB to RAW to generate image analytic for training, where 0013 detail generative adversarial network (GAN), where generative adversarial network is retraining that explicitly incorporates adversarial examples into the training process to improve model robustness).
[Claim 16] The image processing apparatus according to claim 13, further comprising: a signal processing section that executes predetermined signal processing for the RAW data to convert the RAW data to another format; and another data recognition section that executes image recognition processing for an image of the other format to which conversion has been executed by the signal processing section (051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 and paragraph 0069; figure 13 and 0077).
[Claim 17] An information processing method comprising a step of: executing image recognition processing on a basis of an image of RAW data (051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 and paragraph 0069; figure 13 and 0077).
[Claim 18] A program that causes a computer to function as: a RAW data recognition section that executes image recognition processing on a basis of an image of RAW data (051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 and paragraph 0069; figure 13 and 0077).
[Claim 19] An image processing apparatus comprising: an image recognition section to which image data corresponding to an image of a first arrangement according to an arrangement of a pixel array including an imaging element is input, the image recognition section executing image recognition processing for the image data and outputting a recognition processing result (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077), wherein the image recognition section is trained by using the image data corresponding to the image of the first arrangement generated by converting an image of a second arrangement different from the first arrangement (0008 detail convert RGB into raw format and train generative model using the converted data from RGB to RAW; 0029).
[Claim 20] An image processing method of an image processing apparatus including an image recognition section to which image data corresponding to an image of a first arrangement according to an arrangement of a pixel array including an imaging element is input, the image recognition section executing image recognition processing for the image data and outputting a recognition processing result (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077), the image processing method comprising a step of:
by the image recognition section, executing the image recognition processing for the image data and outputting the recognition processing result after execution of learning of the image recognition processing using the image data corresponding to the image of the first arrangement generated by conversion of an image of a second arrangement different from the first arrangement (0008 detail convert RGB (first arrangement) into raw format (second arrangement) and train generative model using the converted data from RGB to RAW; 0029).
[Claim 21] An image conversion apparatus comprising: an image conversion section that converts an RGB image having an R image, a G image, and a B image to an image including another arrangement different from an arrangement of the RGB image output according to an arrangement of a pixel array including an imaging element (0008 detail convert RGB (first arrangement) into raw format (second arrangement) and train generative model using the converted data from RGB to RAW; 0029), wherein the image including the other arrangement is used for learning of an image recognition section used for image inference processing (0056 detail neural network perform inference from raw) based on the image including the other arrangement (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077).
[Claim 22] An image conversion method comprising a step of: converting an RGB image having an R image, a G image, and a B image to an image including another arrangement different from an arrangement of the RGB image output according to an arrangement of a pixel array including an imaging element (0008 detail convert RGB (first arrangement) into raw format (second arrangement) and train generative model using the converted data from RGB to RAW; 0029), wherein the image including the other arrangement is used for learning of an image recognition section used for image inference processing based on the image including the other arrangement (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077).
[Claim 23] An AI network generation apparatus comprising: an image conversion section that converts an input image of a first arrangement to an image of a second arrangement different from the first arrangement and outputs the image of the second arrangement (0008 detail convert RGB (first arrangement) into raw format (second arrangement) and train generative model using the converted data from RGB to RAW; 0029); and an AI network training section that generates a trained AI network by training an AI network by using the image of the second arrangement output from the image conversion section (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077).
[Claim 24] An AI network generation method comprising steps of: converting an input image of a first arrangement to an image of a second arrangement different from the first arrangement (0008 detail convert RGB (first arrangement) into raw format (second arrangement) and train generative model using the converted data from RGB to RAW; 0029) and outputting the image of the second arrangement; and generating a trained AI network by training an AI network by using the output image of the second arrangement (0051 generate image data in a raw format, the generative model trained using the analytic information (from the conversion) with identify patterns (training recognition); figure 10 paragraph 0069; figure 13 and 0077).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
ZHANG et al (US 2023/0222639) detail DATA PROCESSING METHOD, SYSTEM, AND APPARATUS - collect the RAW-RGB dataset, and the collected RAW-RGB dataset (0004).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TSUNG-YIN TSAI whose telephone number is (571)270-1671. The examiner can normally be reached 7am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at (571) 272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSUNG YIN TSAI/Primary Examiner, Art Unit 2656
Prosecution Insights