DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim 1, 2, 5, 9-12 and 15-16 are rejected under 35 U.S.C. 102(a) (2) as being anticipated by Shinya Fujiwara [US 20230081752 A1].
Regarding claim 1, Shinya teaches:
1. A photoelectric conversion device (i.e. The present invention relates to a video control device, a video recording device, a video control method, a video recording method, and a non-transitory computer readable recording medium storing a video control program- ¶0002t) comprising:
a photoelectric conversion element configured to generate an image signal corresponding to an incident light quantity (i.e. The imaging element 102 converts an optical image by the imaging lens system 101 into an analog image signal, and outputs the converted analog image signal to the ADC 103. The imaging element 102 is configured by an imaging element, such as a complementary metal-oxide-semiconductor (CMOS) type image sensor or a charge-coupled device (CCD) type image sensor- ¶0030);
one or more memories storing instructions (i.e. non-transitory computer readable recording medium storing a video control program capable of performing imaging with an appropriate exposure value depending on whether or not pixel data before demosaicing processing is recorded); and one or more processors (i.e. e program causing a processor of the video control device to execute a process- ¶0006) executing the instructions to:
perform visual-recognition image processing to generate an image for visual recognition (i.e. The RAW correction unit 105 corrects the RAW pixel data output from the ADC 103. The correction performed by the RAW correction unit 105 is correction performed on the RAW pixel data before the demosaicing processing, and examples thereof include pixel value correction, defective pixel correction, and shading correction in accordance with a characteristic of the imaging element 102 and the like. The RAW correction unit 105 outputs the corrected RAW pixel data to transitory storage unit 106- ¶0035),
perform image-recognition image processing to generate an image for image recognition (i.e. The development processing unit 107 based on the RAW pixel data and the metadata stored by the transitory storage unit 106 generates a demosaicing video by performing the development processing including the demosaicing processing, and outputs the generated demosaicing video to the monitor 108- ¶0037), and
perform control such that image processing is performed on the image signal generated by the photoelectric conversion element using at least one of the visual-recognition image processing and the image-recognition image processing (i.e. the imaging control unit 104 may automatically control the exposure and the like of the imaging by the imaging unit 119 based on a demosaicing image obtained by the development processing unit 107, which will be described below.- ¶0033).
Regarding claim 2, Shinya teaches all the limitations of claim 1 and Shinya further teaches:
wherein the photoelectric conversion element includes a plurality of pixels of which each includes a photoelectric conversion unit emitting a pulse in response to incidence of photons, a counter counting the number of pulses, and a memory storing a count value of the counter, and wherein the photoelectric conversion element generates an image signal on the basis of a difference of the count value between a start timing of an accumulation period and an end timing of the accumulation period in each photoelectric conversion unit (i.e. The imaging element 102 converts an optical image by the imaging lens system 101 into an analog image signal, and outputs the converted analog image signal to the ADC 103. The imaging element 102 is configured by an imaging element, such as a complementary metal-oxide-semiconductor (CMOS) type image sensor or a charge-coupled device (CCD) type image sensor- ¶0030).
Regarding claim 5, Shinya teaches all the limitations of claim 1 and Shinya further teaches:
wherein the one or more processors further execute the instructions to perform compression for reducing an amount of data of at least the image signal generated by the photoelectric conversion element in the image-recognition image processing (i.e. the development processing unit 107 generates the video data by performing various types of the image processing on the image data generated by the demosaicing processing, and outputs the generated video data to the monitor 108. Examples of the image processing performed after the development processing unit 107 performs the demosaicing processing include processing of gain correction, gamma correction, peripheral light amount falloff correction, color correction, contour enhancement, noise reducing, debayer processing, and compression- ¶0038).
Regarding claim 9, Shinya teaches all the limitations of claim 1 and Shinya further teaches:
wherein the one or more processors further execute the instructions to perform defective pixel correction in the visual-recognition image processing (i.e. The RAW correction unit 105 corrects the RAW pixel data output from the ADC 103. The correction performed by the RAW correction unit 105 is correction performed on the RAW pixel data before the demosaicing processing, and examples thereof include pixel value correction, defective pixel correction, and shading correction in accordance with a characteristic of the imaging element 102 and the like. The RAW correction unit 105 outputs the corrected RAW pixel data to transitory storage unit 106- ¶0035).
Regarding claim 10, Shinya teaches all the limitations of claim 1 and Shinya further teaches:
wherein the one or more processors further execute the instructions to perform black level correction in the image-recognition image processing (i.e. The metadata is used in a case in which development processing of generating the demosaicing image based on the RAW pixel data is performed. Examples of the metadata include a black offset level, a coefficient for converting a RAW value into a color system, a white balance parameter, a lens correction parameter, a color conversion parameter, a gamma correction parameter, a noise correction parameter, a time code, imaging date and time, and a product name- ¶0034).
Regarding claim 11, Shinya teaches all the limitations of claim 1 and Shinya further teaches:
wherein the one or more processors further execute the instructions to allow the visual-recognition image processing and the image-recognition image processing to share a processing part (i.e. see fig. 1).
Regarding claim 12, Shinya teaches all the limitations of claim 1 and Shinya further teaches:
wherein the one or more processors further execute the instructions to allow the shared processing part to use different correction parameters when the visual recognition is performed and when the image recognition is performed.
Regarding claim 15, method claim 15 corresponds to apparatus claim 1, and therefore is also rejected for the same rationale as listed above.
Regarding claim 16, computer-readable medium storing instructions claim 16 corresponds to apparatus claim 1, and therefore is also rejected for the same rationale as listed above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 3-4 are rejected under 35 U.S.C. 103 as being unpatentable over Shinya Fujiwara [US 20230081752 A1] in view of Andrew Dewhurst [US 20180082408 A1].
Regarding claim 3, Shinya teaches all the limitations of claim 1.
However, Shinya does not teach explicitly:
wherein control is performed such that an image signal generated in an image-recognition accumulation period is output between an end of the image-recognition accumulation period and an end of a visual-recognition accumulation period, one full frame period including the image-recognition accumulation period for generating an image-recognition image and the visual-recognition accumulation period for generating a visual-recognition image.
In the same field of endeavor, Andrew teaches:
wherein control is performed such that an image signal generated in an image-recognition accumulation period is output between an end of the image-recognition accumulation period and an end of a visual-recognition accumulation period, one full frame period including the image-recognition accumulation period for generating an image-recognition image and the visual-recognition accumulation period for generating a visual-recognition image (i.e. FIG. 1b shows a schematic representation of timing of processing three exposures of the same scene to produce a multi-exposure high dynamic range image, according to some embodiments- ¶0031).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Shinya with the teachings of Andrew to reduce this storage requirement and significantly reduce bandwidth requirements of multi-exposure HDR camera systems (Andrew- ¶0008).
Regarding claim 4, Shinya teaches all the limitations of claim 1.
However, Shinya does not teach explicitly:
wherein the one or more processors further execute the instructions to generate an image with higher image quality in the visual-recognition image processing than in the image-recognition image processing and to generate an image faster in the image-recognition image processing than in the visual-recognition image processing.
In the same field of endeavor, Andrew teaches:
wherein the one or more processors further execute the instructions to generate an image with higher image quality in the visual-recognition image processing than in the image-recognition image processing and to generate an image faster in the image-recognition image processing than in the visual-recognition image processing (i.e. The first exposure 105 has a longer exposure than the second exposure 110, and the second exposure 110 has a longer exposure than the third exposure 115. A longer exposure causes an exposure to be more sensitive to details in areas of low light and also causes brighter areas to be saturated. Conversely, a shorter exposure causes an exposure to be more sensitive to details in areas of bright light, and also causes loss of detail in darker areas- ¶0040).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Shinya with the teachings of Andrew to reduce this storage requirement and significantly reduce bandwidth requirements of multi-exposure HDR camera systems (Andrew- ¶0008).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Shinya Fujiwara [US 20230081752 A1] and further in view of Thomas M. Carroll [US 5058174 A]:
Regarding claim 6, Shinya teaches all the limitations of claim 5.
However, Shinya does not teach explicitly:
wherein the one or more processors further execute the instructions to reduce the amount of data by adding or addition-averaging pixel values of the plurality of pixels in the compression.
In the same field of endeavor, Thomas teaches:
wherein the one or more processors further execute the instructions to reduce the amount of data by adding or addition-averaging pixel values of the plurality of pixels in the compression (i.e. Compressor 122 combines the RGB color values for each pixel into a single, averaged RGB color value. Each pixel therefore, is assigned a single RGB color value. Compressor 122 then assigns a single RGB color to each subarea by computing an average color value from among the color values assigned each of the 25 pixels within each subarea. Compressor 122, therefore, reduces 192,700 pixels, each having separate R,G,B color values, into 7708 subareas, each having a single R,G,B color value. Each subarea value has a color corresponding to the average of the color values assigned to the 25 pixels within each subarea, to form a compressed image 113 (See FIG. 2) stored in compressed image memory 118- col 3, line 35-48).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Shinya with the teachings of Thomas to generate an image having a preset number of colors (Thomas – Col 5, line 11-12).
Claims 7-8 are rejected under 35 U.S.C. 103 as being unpatentable over Shinya Fujiwara [US 20230081752 A1] and further in view of Tetsu Wada [US 20060274953 A1]:
Regarding claim 7, Shinya teaches all the limitations of claim 5.
However, Shinya does not teach explicitly:
wherein the one or more processors further execute the instructions to reduce the amount of data by thinning out pixels in rows or columns of the photoelectric conversion element in the compression.
In the same field of endeavor, Tetsu teaches:
wherein the one or more processors further execute the instructions to reduce the amount of data by thinning out pixels in rows or columns of the photoelectric conversion element in the compression(i.e. According to this signal processing method, thinned data are generated by thinning, in a checkerboard like manner, image data formed of a plurality of pixels that are arranged as a square; the first square array and the second square array are generated by respectively extracting only the odd-numbered lines and the even-numbered lines from the lines of thinned data in the row direction and in the column direction; and the image compression process is performed separately for the first and the second square arrays- ¶0015).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Shinya with the teachings of Tetsu to reduce the volume of data without the image quality being deteriorated (Tetsu- ¶0015).
Regarding claim 8, Shinya teaches all the limitations of claim 5.
However, Shinya does not teach explicitly:
wherein the one or more processors further execute the instructions to reduce the amount of data by cutting out pixels in a partial area of the photoelectric conversion element in the image-recognition image processing.
In the same field of endeavor, Tetsu teaches:
wherein the one or more processors further execute the instructions to reduce the amount of data by cutting out pixels in a partial area of the photoelectric conversion element in the image-recognition image processing (i.e. According to this signal processing method, thinned data are generated by thinning, in a checkerboard like manner, image data formed of a plurality of pixels that are arranged as a square; the first square array and the second square array are generated by respectively extracting only the odd-numbered lines and the even-numbered lines from the lines of thinned data in the row direction and in the column direction; and the image compression process is performed separately for the first and the second square arrays- ¶0015).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Shinya with the teachings of Tetsu to reduce the volume of data without the image quality being deteriorated (Tetsu- ¶0015).
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Shinya Fujiwara [US 20230081752 A1] and further in view of Vladimir Koifman et al. [US 8184173 B1]:
Regarding claim 13, Shinya teaches all the limitations of claim 1.
However, Shinya does not teach explicitly:
wherein the photoelectric conversion element includes an avalanche photodiode.
In the same field of endeavor, Vladimir teaches:
wherein the photoelectric conversion element includes an avalanche photodiode (i.e. Digital cameras include a two-dimensional pixel array. Each pixel includes a light sensitive element that converts photons to an analog signal. The light sensitive elements can include photodiodes, phototransistors, photo-gates, hole accumulation diodes, pinned diodes, avalanche diodes, buried accumulation and transfer layer devices-Col 1, line 18-23).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Shinya with the teachings of Vladimir since Various prior art pixels are known and the most commonly used pixels are either CCD pixels or CMOS pixels (Vladimir- Col 1, line 24-25).
Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Shinya Fujiwara [US 20230081752 A1].
Regarding claim 14, Shinya teaches:
14. A movable apparatus (i.e. The present invention relates to a video control device, a video recording device, a video control method, a video recording method, and a non-transitory computer readable recording medium storing a video control program- ¶0002t) comprising:
a photoelectric conversion element configured to generate an image signal corresponding to an incident light quantity (i.e. The imaging element 102 converts an optical image by the imaging lens system 101 into an analog image signal, and outputs the converted analog image signal to the ADC 103. The imaging element 102 is configured by an imaging element, such as a complementary metal-oxide-semiconductor (CMOS) type image sensor or a charge-coupled device (CCD) type image sensor- ¶0030);
one or more memories storing instructions (i.e. non-transitory computer readable recording medium storing a video control program capable of performing imaging with an appropriate exposure value depending on whether or not pixel data before demosaicing processing is recorded); and one or more processors (i.e. e program causing a processor of the video control device to execute a process- ¶0006) executing the instructions to:
perform visual-recognition image processing to generate an image for visual recognition (i.e. The RAW correction unit 105 corrects the RAW pixel data output from the ADC 103. The correction performed by the RAW correction unit 105 is correction performed on the RAW pixel data before the demosaicing processing, and examples thereof include pixel value correction, defective pixel correction, and shading correction in accordance with a characteristic of the imaging element 102 and the like. The RAW correction unit 105 outputs the corrected RAW pixel data to transitory storage unit 106- ¶0035),
perform image-recognition image processing to generate an image for image recognition (i.e. The development processing unit 107 based on the RAW pixel data and the metadata stored by the transitory storage unit 106 generates a demosaicing video by performing the development processing including the demosaicing processing, and outputs the generated demosaicing video to the monitor 108- ¶0037), and
perform control such that image processing is performed on the image signal generated by the photoelectric conversion element using at least one of the visual-recognition image processing and the image-recognition image processing and control of an operation of the movable apparatus (i.e. the imaging control unit 104 may automatically control the exposure and the like of the imaging by the imaging unit 119 based on a demosaicing image obtained by the development processing unit 107, which will be described below.- ¶0033).
However, Shinya does not teach explicitly:
movable.
It would have been obvious to one having ordinary skill in the art at the time the invention was filed to make a device movable/portable, since it has been held that making an old device portable or movable without producing any new and unexpected result involves only routine skill in the art. In re Lindberg, 93 USPQ 23 (CCPA 1952).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CLIFFORD HILAIRE whose telephone number is (571)272-8397. The examiner can normally be reached 5:30-1400.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, SATH V PERUNGAVOOR can be reached at (571)272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CLIFFORD HILAIRE
Primary Examiner
Art Unit 2488
/CLIFFORD HILAIRE/Primary Examiner, Art Unit 2488