Detailed Action
1. Claims 1-20 are pending in this Application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 3 7 CFR 1.114.
3. A request for continued examination under 37 CFR 1.1 14, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.1 14. Applicant's submission filed on 0218/2026 has been entered.
Information Disclosure Statement
4. The information disclosure statement filed on 06/25/2026 fails to comply with 37 CFR 1.98(a)(3)(i) because it does not include the reference listed on it. It has been placed in the application file, but the information referred to therein has not been considered.
Response to amendment
5. Applicant’s response to the last Office Action filed on 04/01/2026 has been entered and made of record.
6. Claims1 and 18 have been amended.
.
Response to Argument
7. The Applicant’s argument filed 06/25/2026 is fully consider. For Examiner response see discussion below.
8. The Applicant’s has amended claims 1 and 18 as follows: “wherein usage of image processing using the nonvisible light image data differs depending on [[(a)]]] conditions under which the nonvisible light image data is generated by the imaging unit and .”, and substantially argue that the applied prior arts do not teach the added limitation.
After further consideration Examiner find a reference (JP2008299645A to KITAMURA et al., IDS item)that specifically teach the above limitation. Specifically KITAMURA et al., teaches image processing based on the time is day or night. In day time operation the infrared light stays off because the sun provides enough light, and the camera takes a normal visible-light picture of the license plate. On the other hand in the night time the system turns on the infrared light to light up the vehicle, the camera takes an infrared picture of the license plate, and the system saves the picture and reads the letters and numbers (see claims 1 and 2; [0019]-[0027]) . Thus, from the above explanation the limitation “wherein usage of image processing using the nonvisible light image data differs depending on conditions under which the nonvisible light image data is generated by the imaging unit and the processing unit determines to perform image processing using the nonvisible light image data” corresponds to at night, the system activates an infrared illuminator to illuminate the vehicle. Upon detecting nighttime conditions, the camera captures an infrared image of the license plate, which the system then stores and processes via optical character recognition (OCR) to extract the alphanumeric characters.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
9. Claims 1-6,8,12-15 and 17-20 are rejected under 35 U.S.C. 103 as being unpatentable over OTSUHI HIRONAO (hereafter OTSU), JP-2010154266A, pub. 07/08/2010, in view of KITAMURA et al.,(hereafter KITAMURA), JP2008299645A, pub. 12/11/2008
As to claim 1, OTSU teaches An imaging control apparatus (Abstract: an infrared irradiation type image pickup device ) comprising: at least one processor; and a memory coupled to the at least one processor ([0074], the imaging device includes a CPU, a main storage device such as a RAM, and a computer-readable storage medium in the system that stores a program) wherein the memory has instructions that, when executed by the at least one processor, perform operations as a plurality of unites ([0074], the CPU then reads out the control program stored in the storage medium and executes information processing and calculation processing) comprising :
a control unit configured to acquire visible light image data and nonvisible light image data through imaging by an imaging unit ([009], [0018]-[0019], an imaging condition control unit that causes the imaging unit to generate a visible light imaging signal under visible light imaging conditions in which the infrared cut filter is placed at the blocking position, and that causes the imaging unit to generate an infrared mixed imaging signal under
infrared mixed imaging conditions);
a processing unit configured to perform image processing using at least one of the visible light image data and the nonvisible light image data ([00], [0027], an image signal consisting of visible light components, hereinafter, as a visible light image signal RGBxy. The RGBxy is generated by the imaging unit 4 and output to the white balance processing unit 10 and. The white balance processing unit 10 performs white balance processing on the visible light imaging signal RGBxy to generate a white balanced imaging signal wRGBxy. The color-matched image signal processor process the adding the infrared calculation image signal [Axy] to the lightness L component the visible image) ; and
a determining unit configured to determine an imaging scene, and wherein the control unit controls whether or not to perform imaging for the nonvisible light image data by the imaging unit and image processing using the nonvisible light image data in the processing unit according to a determination result of the imaging scene (Claim 2, [0049], the infrared illumination type imaging device according to claim 2, further comprising: a warning unit that issues a warning to an operator when the color change state is detected or when it is determined that the scene has changed; and an operation unit that waits for control switching from the infrared mixed imaging condition to the visible light imaging condition until an operation is performed by the operator);and
wherein usage of image processing using the nonvisible light image data differs depending on (a) conditions under which the nonvisible light image data is generated by the imaging unit ( page 6 last two pars., the image processing unit 14 is not limited to the resampling process as described above, but various image processes such as luminance gradation correction, detail correction, various noise cancellation filters, color correction, and resolution conversion . The corresponded is derived based on the specification of the application.(see par. [0025] of PGPUB) ).
However, it is noted that OTSU does not specifically teach “wherein usage of image processing using the nonvisible light image data differs depending on conditions under which the processing unit determines to perform image processing using the nonvisible light image data”
On the other hand KITAMURA teaches wherein usage of image processing using the nonvisible light image data differs depending on conditions under which the processing unit determines to perform image processing using the nonvisible light image data (claims 1 and 2; [0019]-[0027], A license plate reading device comprising: an image input unit capable of inputting an infrared image using infrared light and a visible image using visible light as a captured image of an area including the license plate of a vehicle; an illumination unit that irradiates with infrared illumination light; a day/night determination unit that determines day or night based on the brightness of the license plate area; a license plate reading unit that recognizes the character information of the license plate from at least one of the infrared image and the visible image; a reading result determination unit that determines the license plate reading result in the license plate reading unit; and a reading result output unit that outputs the license plate reading result including the license plate information, wherein the reading result determination unit, based on the determination result of the day/night determination unit, uses the license plate reading result from the visible image in the license plate reading unit if the license plate area is brighter than a predetermined condition and is determined to be daytime. Wherein the image input unit inputs a visible image when the license plate portion is brighter than predetermined conditions and is determined to be daytime, based on the determination result of the day/night determination unit, and inputs an infrared image when the license plate portion is darker than predetermined conditions and is determined to be nighttime.)
A person of ordinary skill in the art would have been motivated, before the effective filing date of the claimed invention, to modify Otsu by incorporating the day/night determination unit taught by Kitamura, which determines day or night based on the brightness surrounding an object to be imaged.
The motivation to combine these references would have been to enable an Otsu-based system to automatically evaluate the brightness around the object and switch between ambient light and infrared light for image capture.
As to claim 2, OTSU teaches the imaging unit includes a first imaging unit for visible light and a second imaging unit for nonvisible light (claim1,[0009], imaging unit to generate a visible light imaging signal under visible light imaging conditions in which the infrared cut filter is located at the blocking position, and that causes the imaging unit to generate an infrared mixed imaging signal under infrared mixed imaging conditions)
As to claim 3, OTSU teaches the instructions when executed by the at least one processor further perform operations as a separating unit configured to separate the visible light image data and the nonvisible light image data from image data generated by the imaging unit ([0009], a separation unit that separates the infrared mixed imaging signal into a visible light calculation imaging signal and an infrared mixed imaging signal based on the ratio and/or difference of the signal amounts of the visible light imaging signal and the infrared mixed imaging signal).
As to claim 4, OTSU teaches the determining unit determines the imaging scene according to a setting by a user (claim 3, [0018], The imaging condition control unit 7 controls the infrared irradiating unit 1 and the filter driving unit 6 to switch between imaging conditions of visible light and infrared mixed imaging conditions and an operation unit that waits for controls witching from the infrared mixed imaging condition to the visible light imaging condition until an operation is performed by the operator).
As to claim 5, OTSU teaches the determining unit determines the imaging scene according to a result of object recognition using at least one of the visible light image data and the nonvisible light image data( [0061], FIG. 6 shows a case where gray world control is performed on an infrared mixed image signal RGBAxy obtained by capturing an image of the Japanese flag).
As to claim 6, KITAMURA teaches the determining unit selects image data to be used for the object recognition from the visible light image data and the nonvisible light image data according to the imaging scene ( calims1 and 2, day/night determination unit that determines day or night based on the brightness of the license plate area, image input unit inputs a visible image when the license plate portion is brighter than predetermined conditions and is determined to be daytime, based on the determination result of the day/night determination unit, and inputs an infrared image when the license plate portion is darker than predetermined conditions and is determined to be nighttime).
A person of ordinary skill in the art would have been motivated, before the effective filing date of the claimed invention, to modify Otsu by incorporating the day/night determination unit taught by Kitamura, which determines day or night based on the brightness surrounding an object to be imaged.
The motivation to combine these references would have been to enable an Otsu-based system to automatically evaluate the brightness around the object and switch between ambient light and infrared light for image capture.
As to claim 8, OTSU teaches the determining unit determines the imaging scene using luminance information acquired from at least one of the visible light image data and the nonvisible light image data ([0037], For example, a first luminance signal Y1xy is generated by resampling the color-matched imaging signal c[RGBxy], and the infrared calculation imaging signal [Axy] is used as a second luminance signal Y2xy. These luminance signals Y1xy and Y2xy are then synthesized to generate the luminance signal Yxy).
As to claim 12, KITAMURA teaches the control unit controls whether or not to emit a nonvisible light source that irradiates nonvisible light onto an object according to a determination result of the imaging scene ([0019], his image input unit 1 is capable of capturing both visible and infrared images. For example, it may be equipped with two cameras, one for capturing visible images and one for capturing infrared images, allowing for simultaneous capture of both visible and infrared images. Alternatively, it may be a single camera that can switch between modes to capture both visible and infrared images).
A person of ordinary skill in the art would have been motivated, before the effective filing date of the claimed invention, to modify Otsu by incorporating the day/night determination unit taught by Kitamura, which determines day or night based on the brightness surrounding an object to be imaged.
The motivation to combine these references would have been to enable an Otsu-based system to automatically evaluate the brightness around the object and switch between ambient light and infrared light for image capture.
As to claim 13, OTSU teaches in a case where the nonvisible light image data is generated and the image processing using the nonvisible light image data is executed ([0044], The operator sees the flashing and lit warning display and determines whether or not to execute control switching from the infrared mixed imaging condition to the visible light imaging condition.), the control unit records the nonvisible light image data in another memory according to a setting by a user([0074], imaging device includes a CPU, a main storage device such as a RAM, and a computer-readable storage medium in the system that stores a program for realizing all or part of the image processing. Thus, infrared image data and mixed imaging condition are the storage devise).
As to claim 14, OTSU teaches An image pickup apparatus comprising: an imaging unit; and the imaging control apparatus according to claim 1(Abstract).
As to claim 15, OTSU teaches a display unit configured to display to a user that the nonvisible light image data is generated and image processing using the nonvisible light image data is performed ([0044], The operator sees the flashing and lit warning display and determines whether or not to execute control switching from the infrared mixed imaging condition to the visible light imaging condition).
As to claim 17, OTSU teaches a display unit configured to display to a user that a nonvisible light source that irradiates nonvisible light onto an object is lit (([0044], The operator sees the flashing and lit warning display and determines whether or not to execute control switching from the infrared mixed imaging condition to the visible light imaging condition).
Claim 18 is rejected the same as claim 1 except claim 18 is directed to a method claim. All the limitations of claim 18 are addressed in claim 1. Thus, argument analogous to that presented above for claim 1 is applicable to claim 18.
As to claim 19, OTSU teaches A non-transitory computer-readable storage medium storing a program that causes a computer to execute the imaging control method according to claim 18([088], In this way, the imaging device according to an embodiment of the present invention may be realized by software processing, and if realized by software processing, changes or additions to the processing content can be easily accommodated by rewriting the program).
As to claim 20, the combination of OTSU and KITAMURA teaches one or more of the following conditions is satisfied:
(2) the usage of image processing using the nonvisible light image data differs depending on whether night view imaging is being performed (KITAMURA, claims 1 and 2; [0019]-[0027]);
(3) the usage of image processing using the nonvisible light image data differs depending on a determination regarding an imaging light source ( KITAMURA : claim 6, A license plate reading device according to claim 1, comprising a filter attachment/detachment unit in the imaging optical system of the image input unit for attaching and detaching a filter that blocks the visible light band or transmits only the infrared light band, wherein the filter attachment/detachment unit removes the filter when the license plate portion is brighter than a predetermined condition and is determined to be daytime, and inserts the filter when the license plate portion is darker than a predetermined condition and is determined to be nighttime.);
(5) the usage of image processing using the nonvisible light image data differs depending upon whether noise reduction is being used (OTSU: page 6 last two pars., the image processing unit 14 is not limited to the resampling process as described above, but various image processes such as luminance gradation correction, detail correction, various noise cancellation filters, color correction, and resolution conversion);
10 Claims 7, 10 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over OTSU, JP-2010154266A, in view of KITAMURA, JP2008299645A, further in view of YONISHI OSAMU ( hereafter YONISHI), KR 20190079574A, pub. 07/05/2019.
Regarding claim 7, while modified OTSU, teaches the limitation of claim 5, but fails to teach the limitation of claim 7.
On the other hand YONISHI teaches the determining unit uses a learning model for the object recognition, and wherein the determining unit uses a visible light image learning model in a case where the image data for the object recognition is the visible light image data, and the determining unit uses a nonvisible light image learning model as the learning model in a case where the image data for the object recognition is the nonvisible light image data(Page 7, 2nd par.,- 3rd par., as discussed in claim 6 above YONISHI teaches a case when an object
is detected in the infrared light image, it may be difficult to determine whether the infrared light image should be converted into a visible light image. In such a case,
machine learning is applied to the object determining process, and the type of object is determined based on features such as shape and size. Thereafter, the infrared light image may be switched to the visible light image only when an object of a specific detection level or higher is specified. The "detection level" indicates the degree to which an object should be monitored).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a machine learning technique configured to detected object using visible light image or infrared image based on features such as shape size of the object to be detected and also based on the weather conditions taught by YONISHI into modified OTSU.
The suggestion/motivation for doing so would have been to allow user of modified OTSU to detected object more efficiently by utilizing learned visible light image data and leaned infrared image light which are generated by training a large number of image data using ML technique.
As to claim 10, YONISHI teaches the determining unit determines the imaging scene using object distance information(page 6 par. 3, page 2 par., 2., the infrared camera is suitable for monitoring at a distance. In comparison with a visible light image, it is possible to detect an object at a long distance in an ultraviolet light image. Thus, a distance information is utilized when detecting an object using infrared camera).
As to claim 16, OTSU teaches display unit configured to prompt a user to execute image processing, visible image data and nonvisible light image data (this limitation discussed in claim 1 above);
however, it is noted that modified OTSU does not specifically under line section of the limitation of “a display unit configured to prompt a user to execute image processing using the nonvisible light image data in a case where generation of the nonvisible light image data and image processing using the nonvisible light image data is not performed “
On the other hand YONISHI teaches a display unit configured to prompt a user to execute image processing using the nonvisible light image data in a case where generation of the nonvisible light image data and image processing using the nonvisible light image data is not performed (Page 7, 1st- par.,- 3rd par., when the visible light image is always distributed, the object cannot be detected when the photographing condition is bad. According to the above-described embodiment, it is possible to provide an image suitable for monitoring difficult to be affected by weather condition. In this situation visible light image may be switched to the infrared light only when an object of a specific detection level or higher is specified).
11. Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over OTSU,
JP2010154266A, in view of KITAMURA, JP2008299645A, further in view of ZHANG et al., (hereafter, ZHANG), WO 2021217643 A, pub. 11/04/2021.
Regarding claim 9, while modified OTSU, teaches the limitation of claim 1, but fails to teach the limitation of claim 9.
On the other hand ZHANG the determining unit determines the imaging scene using motion vector information acquired from at least one of the visible light image data and the nonvisible light image data(page 7 4th par., For example, the relative position of a visible light sensor and the infrared sensor is fixed, and the two respectively collect visible images and infrared images in the same scene. Since the relative positions of the two are fixed, their global motion vectors are the same. Since the resolution of the visible light image is higher, the motion vector determined from the visible light image will be more accurate. Therefore, the motion vector of the visible light image can be combined to assist in determining the motion vector of the infrared image and its reference image, so that the determined motion vector is more accurate).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method of generating a motion vector by combining visible light and infrared (IR) images taught by ZHANG into modified OTSU.
The suggestion/motivation for doing so would have been to allow user of modified OTSU to generate a robust object detection method by combining rich texture and detail of the object obtained from visible images data of the object, and thermal radiation data of the object obtained from the infrared image data of the object.
12. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over OTSU,
JP-2010154266A, in view of KITAMURA, JP2008299645A , further in view of IKEDA, JUN (hereafter IKEDA), WO 2010109919A, pub. 09/30/2010.
As to claim 11, KITAMURA teaches the control unit controls whether or not to generate the nonvisible light image data and whether or not to execute image processing using the nonvisible light image data according to a determination result of the imaging scene ( calims1 and 2, day/night determination unit that determines day or night based on the brightness of the license plate area, image input unit inputs a visible image when the license plate portion is brighter than predetermined conditions and is determined to be daytime, based on the determination result of the day/night determination unit, and inputs an infrared image when the license plate portion is darker than predetermined conditions and is determined to be nighttime);
however, it is noted that the modified OTSU does not specifically teaches the underline section of the limitation “to execute image processing using the nonvisible light image data according to a determination result of a shake detection result of an image pickup apparatus”.
On the other hand JUN teaches to execute image processing using the nonvisible light image data according to a determination result of a shake detection result of an image pickup apparatus (page 5 last paragraph and page 17 last two paragraphs, the imaging device 9 can detect the shake of the subject from the infrared video signal obtained by irradiating the subject with infrared light that allows the subject motion detection unit 23 to clearly shoot the subject even in a dark place such as at night. Further the camera shake can be detected from the captured infrared video signal, and the camera shake in the visible video signal is corrected based on the detected motion information of the subject)
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate a method of detecting and correcting camera shake in the visible video signal by detect the shake of the subject from the infrared video signal taught JUN into modified OTSU.
The suggestion/motivation for doing so would have been to allow user of modified OTSU to generate visible images data by removing the artifact generated due to camera shake, and generate a robust object detection method by combining rich texture and detail of the object obtained.
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday-Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR. Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.
/MEKONEN T BEKELE/Primary Examiner, Art Unit 2699
Prosecution Insights