Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 8-12 and 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over Shirai et al. (US PGPUB 20120187275) in view of Hu et al. (US PGPUB 20220067544).
[Claim 1]
Shirai teaches an image capturing system including an image capturing apparatus and a lighting device (FIG. 2) , which are communicably connected to each other (Paragraph 55), wherein the image capturing apparatus comprises:
an image sensor (camera 141); and
a first control unit (PC 2) configured to control a charge accumulation time in the image sensor (Paragraph 55, The PC 2 sends a signal designating a frame rate to the camera 141, and sends a PWM frequency instruction for the illumination device 142 to the illumination control device 152. Paragraph 62, Here, an example is shown where the frame period (=the reciprocal of the frame rate) is set substantially equal to the exposure time),
wherein the lighting device comprises:
a light emission section (142); and
a second control unit (152) configured to control light emission from the light emission section by performing PWM control (Paragraph 55, The PC 2 sends a signal designating a frame rate to the camera 141, and sends a PWM frequency instruction for the illumination device 142 to the illumination control device 152),
wherein the first control unit (PC 2) determines a frequency in the PWM control, based on the charge accumulation time, and notifies the determined frequency to the second control unit (Paragraph 72, In the present embodiment, a PWM-controlled LED is used as the illumination device 142, and the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100 fps. This is done by PC 2 and is notified to the illumination control device 152 as explained in Paragraph 55 and fig. 2), and
wherein the second control unit (152) controls light emission from the light emission section at the frequency in the PWM control, which is notified from the first control unit (Paragraph 75, FIG. 12 is basically the same as FIG. 11 but different in that the emission frequency of the illumination device 142 is set to 600 Hz. As can be seen from the upper row of FIG. 12, when the frame rate of the camera 141 is 60 fps, the amount of light by which the workpiece 3 is irradiated by the illumination device 142 during one exposure time of the camera 141 is an amount corresponding to ten pulses of the illumination device 142. When the frame rate of the camera 142 is 50 fps, an amount of light received corresponds to twelve pulses of the illumination device 142. This is done by PC 2 and is notified to the illumination control device 152 as explained in Paragraph 55 and fig. 2).
Shirai teaches an image capturing apparatus and a lighting device but fails to teach an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions.
However Hu teaches a computing device 218 may include a computer readable medium 224. The computer readable medium 224 may be implemented using any suitable medium, including non-transitory computer readable media. Examples include memory, random access memory (RAM), read only memory (ROM), volatile or non-volatile memory, hard drive, solid state drives, or other storage. The computer readable medium 224 may be accessible to the processor(s) 206 and/or memory controller 220. The computer readable medium 224 may be encoded with executable instructions 228. The executable instructions 228 may be executed by the processor(s) 206. For example, the executable instructions 228 may cause the processor(s) 206 to analyze an acquired image to extract a measured spectrum from the image. In some examples, the executable instructions 228 may cause the processor(s) 206 to correct the measured spectrum extracted from an image or acquired from spectrometer 210, for example, to correct for effects of ambient light. In some examples, the executable instructions 228 may cause the processor(s) 206 to provide commands or other control signals to the light source 208, the camera 212, the display 204, spectrometer 210, and/or other components of the computing device 218, such as the memory controller 220 (Paragraph 18).
Therefore taking the combined teachings of Shirai and Hu, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions in order to easily change and implement the instructions automatically or manually based on the user’s needs and experimental goals thereby making the process more efficient.
[Claim 2]
Shirai teaches wherein assuming that the frequency in the PWM control is represented by "F", the charge accumulation time is represented by "t", and a natural number is represented by "N", the frequency in the PWM control is determined by calculating F = 1/(t X N). [In Paragraph 72, fig. 11 and 12, the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100. So t=120 or 100 and F=480 or 600 Hz and 600 = 1/(120x5). 480=1/(120X4) and so F=1/(t X N)].
[Claim 3]
Shirai teaches wherein the instructions are executed to further function as a detection unit (PC 2) configured to detect flicker, and wherein in a case where flicker is not detected by the detection unit, the frequency in the PWM control is determined by calculating F = 1/(t X N) (Paragraph 71, Illumination device is off and the In Paragraph 72, fig. 11 and 12, the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100. So t=120 or 100 and F=480 or 600 Hz and 600 = 1/(120x5). 480=1/(120X4)),
whereas in a case where flicker is detected by the detection unit, the frequency in the PWM control is determined to be the same frequency as a frequency of the detected flicker (Paragraph 69, the exposure time of the camera 141 is adjusted according to the lighting frequency of the interior lamp (the one . This adjusting method is shown in FIG. 9. FIG. 9 is basically the same as FIG. 5, but different in that the exposure time of the camera is set to 1/60 s when the power source frequency is 60 Hz while it is set to 1/50 s when the power source frequency is 50 Hz. By adjusting the exposure time flexibly according to the power source frequency, it is possible to acquire images with no flicker due to external diffused light. Emission frequency is set in fig. 9 to be an integer multiple of frame rate).
[Claim 4]
Shirai teaches wherein in a case where flicker is detected by the detection unit, the charge accumulation time is set to a natural number multiple of a period of the detected flicker (Paragraph 69, To solve this problem, the present invention employs adjusting the exposure time of the camera 141 according to the lighting frequency of the interior lamp. This adjusting method is shown in FIG. 9. FIG. 9 is basically the same as FIG. 5, but different in that the exposure time of the camera is set to 1/60 s when the power source frequency is 60 Hz while it is set to 1/50 s when the power source frequency is 50 Hz. By adjusting the exposure time flexibly according to the power source frequency, it is possible to acquire images with no flicker due to external diffused light. Note that although in FIG. 9, the exposure time of the camera is twice as long as the time of one period of the lighting frequency of the interior lamp, it needs not be twice as long but needs only to be an integral multiple.)
[Claim 5]
Shirai teaches wherein in a case where flicker is detected by the detection unit, when it is impossible to set the frequency in the PWM control to the same frequency as the frequency of the detected flicker, the frequency in the PWM control is determined by calculating F = 1/(t X N) ((Paragraph 71, Illumination device is off and the In Paragraph 72, fig. 11 and 12, the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100. So t=120 or 100 and F=480 or 600 Hz and 600 = 1/(120x5). 480=1/(120X4)).
[Claim 8]
Shirai teaches an image capturing system including an image capturing apparatus and a lighting device (FIG. 2) , which are communicably connected to each other, wherein the image capturing apparatus comprises:
an image sensor (camera 141); and
a first control unit (PC 2) configured to control a charge accumulation time in the image sensor (Paragraph 55, The PC 2 sends a signal designating a frame rate to the camera 141, and sends a PWM frequency instruction for the illumination device 142 to the illumination control device 152), wherein the lighting device comprises:
a light emission section (142); and
a second control unit (152) configured to control light emission from the light emission section by performing PWM control (Paragraph 55, The PC 2 sends a signal designating a frame rate to the camera 141, and sends a PWM frequency instruction for the illumination device 142 to the illumination control device 152),
wherein the first control unit (PC 2) notifies the charge accumulation time to the second control unit (Paragraph 72, In the present embodiment, a PWM-controlled LED is used as the illumination device 142, and the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100 fps. This is done by PC 2 and is notified to the illumination control device 152 as explained in Paragraph 55 and fig. 2), and
wherein the second control unit (152) determines a frequency in the PWM control, based on the charge accumulation time, and controls light emission from the light emission section (Paragraph 75, FIG. 12 is basically the same as FIG. 11 but different in that the emission frequency of the illumination device 142 is set to 600 Hz. As can be seen from the upper row of FIG. 12, when the frame rate of the camera 141 is 60 fps, the amount of light by which the workpiece 3 is irradiated by the illumination device 142 during one exposure time of the camera 141 is an amount corresponding to ten pulses of the illumination device 142. When the frame rate of the camera 142 is 50 fps, an amount of light received corresponds to twelve pulses of the illumination device 142).
Shirai teaches an image capturing apparatus and a lighting device but fails to teach an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions.
However Hu teaches a computing device 218 may include a computer readable medium 224. The computer readable medium 224 may be implemented using any suitable medium, including non-transitory computer readable media. Examples include memory, random access memory (RAM), read only memory (ROM), volatile or non-volatile memory, hard drive, solid state drives, or other storage. The computer readable medium 224 may be accessible to the processor(s) 206 and/or memory controller 220. The computer readable medium 224 may be encoded with executable instructions 228. The executable instructions 228 may be executed by the processor(s) 206. For example, the executable instructions 228 may cause the processor(s) 206 to analyze an acquired image to extract a measured spectrum from the image. In some examples, the executable instructions 228 may cause the processor(s) 206 to correct the measured spectrum extracted from an image or acquired from spectrometer 210, for example, to correct for effects of ambient light. In some examples, the executable instructions 228 may cause the processor(s) 206 to provide commands or other control signals to the light source 208, the camera 212, the display 204, spectrometer 210, and/or other components of the computing device 218, such as the memory controller 220 (Paragraph 18).
Therefore taking the combined teachings of Shirai and Hu, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions in order to easily change and implement the instructions automatically or manually based on the user’s needs and experimental goals thereby making the process more efficient.
[Claims 9-12]
These claims are similar to claims 2-5 and are analyzed and rejected based upon claims 2-5.
[Claim 15]
Shirai teaches an image capturing apparatus comprising:
an image sensor (fig. 2, camera 141);
a communication unit (fig. 2, illumination control device 152) configured to perform communication with a lighting device (Paragraph 152);
a determination unit (PC 2) configured to determine, assuming that the charge accumulation time in the image sensor is represented by "t", a natural number is represented by "N", and the frequency in the PWM control at a time when the lighting device controls light emission by performing the PWM control is represented by "F", the frequency in the PWM control by calculating F = 1/(t X N) [In Paragraph 72, fig. 11 and 12, the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100. So t=120 or 100 and F=480 or 600 Hz and 600 = 1/(120x5). 480=1/(120X4) and so F=1/(t X N)], and
a notification unit (PC 2) configured to notify the frequency determined by the determination unit by using the communication unit to the lighting device (Paragraph 72, In the present embodiment, a PWM-controlled LED is used as the illumination device 142, and the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100 fps. This is done by PC 2 and is notified to the illumination control device 152 as explained in Paragraph 55 and fig. 2).
Shirai teaches an image capturing apparatus and a lighting device but fails to teach an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions.
However Hu teaches a computing device 218 may include a computer readable medium 224. The computer readable medium 224 may be implemented using any suitable medium, including non-transitory computer readable media. Examples include memory, random access memory (RAM), read only memory (ROM), volatile or non-volatile memory, hard drive, solid state drives, or other storage. The computer readable medium 224 may be accessible to the processor(s) 206 and/or memory controller 220. The computer readable medium 224 may be encoded with executable instructions 228. The executable instructions 228 may be executed by the processor(s) 206. For example, the executable instructions 228 may cause the processor(s) 206 to analyze an acquired image to extract a measured spectrum from the image. In some examples, the executable instructions 228 may cause the processor(s) 206 to correct the measured spectrum extracted from an image or acquired from spectrometer 210, for example, to correct for effects of ambient light. In some examples, the executable instructions 228 may cause the processor(s) 206 to provide commands or other control signals to the light source 208, the camera 212, the display 204, spectrometer 210, and/or other components of the computing device 218, such as the memory controller 220 (Paragraph 18).
Therefore taking the combined teachings of Shirai and Hu, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions in order to easily change and implement the instructions automatically or manually based on the user’s needs and experimental goals thereby making the process more efficient.
[Claim 16]
Shirai teaches an image capturing apparatus comprising:
an image sensor (fig. 2, camera 141);
a light emission section (illumination device 151) configured to emit light toward an object (Paragraph 55); and
a control unit (PC 2) configured to control light emission from the light emission section by performing PWM control and control a charge accumulation time in the image sensor (Paragraph 55, The PC 2 sends a signal designating a frame rate to the camera 141, and sends a PWM frequency instruction for the illumination device 142 to the illumination control device 152. Paragraph 62, Here, an example is shown where the frame period (=the reciprocal of the frame rate) is set substantially equal to the exposure time),
wherein assuming that the charge accumulation time is represented by "t", a natural number is represented by "N", and the frequency in the PWM control is represented by "F", the control unit determines the frequency in the PWM control by calculating F = 1/(t X N) [In Paragraph 72, fig. 11 and 12, the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100. So t=120 or 100 and F=480 or 600 Hz and 600 = 1/(120x5). 480=1/(120X4) and so F=1/(t X N)].
Shirai teaches an image capturing apparatus and a lighting device but fails to teach an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions.
However Hu teaches a computing device 218 may include a computer readable medium 224. The computer readable medium 224 may be implemented using any suitable medium, including non-transitory computer readable media. Examples include memory, random access memory (RAM), read only memory (ROM), volatile or non-volatile memory, hard drive, solid state drives, or other storage. The computer readable medium 224 may be accessible to the processor(s) 206 and/or memory controller 220. The computer readable medium 224 may be encoded with executable instructions 228. The executable instructions 228 may be executed by the processor(s) 206. For example, the executable instructions 228 may cause the processor(s) 206 to analyze an acquired image to extract a measured spectrum from the image. In some examples, the executable instructions 228 may cause the processor(s) 206 to correct the measured spectrum extracted from an image or acquired from spectrometer 210, for example, to correct for effects of ambient light. In some examples, the executable instructions 228 may cause the processor(s) 206 to provide commands or other control signals to the light source 208, the camera 212, the display 204, spectrometer 210, and/or other components of the computing device 218, such as the memory controller 220 (Paragraph 18).
Therefore taking the combined teachings of Shirai and Hu, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions in order to easily change and implement the instructions automatically or manually based on the user’s needs and experimental goals thereby making the process more efficient.
[Claim 17]
Shirai teaches a lighting device (fig. 2) comprising:
a light emission section (illumination device 142); and
a control unit (PC 2) configured to control the light emission section by performing PWM control (Paragraph 55, The PC 2 sends a signal designating a frame rate to the camera 141, and sends a PWM frequency instruction for the illumination device 142 to the illumination control device 152. Paragraph 62, Here, an example is shown where the frame period (=the reciprocal of the frame rate) is set substantially equal to the exposure time);
a communication unit (part of the PC 2) configured to perform communication with an image capturing apparatus (Paragraph 55, In the present embodiment, the illumination device 142 is a PWM (Pulse Width Modulation)-controlled LED. The PC 2 controls the in-focus position of the camera 141 through the position control system 151. The PC 2 sends a signal designating a frame rate to the camera 141); and
an acquisition unit (part of camera 141) configured to acquire a charge accumulation time set to an image sensor included in the image capturing apparatus, by using the communication unit, wherein assuming that the charge accumulation time is represented by "t", a natural number is represented by "N", and the frequency in the PWM control is represented by "F", the control unit determines the frequency in the PWM control by calculating F = 1/(t X N) [In Paragraph 72, fig. 11 and 12, the PWM frequency is set to an integral multiple of 600 Hz which is the least common multiple of 120 and 100. So t=120 or 100 and F=480 or 600 Hz and 600 = 1/(120x5). 480=1/(120X4) and so F=1/(t X N)].
Shirai teaches an image capturing apparatus and a lighting device but fails to teach an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions.
However Hu teaches a computing device 218 may include a computer readable medium 224. The computer readable medium 224 may be implemented using any suitable medium, including non-transitory computer readable media. Examples include memory, random access memory (RAM), read only memory (ROM), volatile or non-volatile memory, hard drive, solid state drives, or other storage. The computer readable medium 224 may be accessible to the processor(s) 206 and/or memory controller 220. The computer readable medium 224 may be encoded with executable instructions 228. The executable instructions 228 may be executed by the processor(s) 206. For example, the executable instructions 228 may cause the processor(s) 206 to analyze an acquired image to extract a measured spectrum from the image. In some examples, the executable instructions 228 may cause the processor(s) 206 to correct the measured spectrum extracted from an image or acquired from spectrometer 210, for example, to correct for effects of ambient light. In some examples, the executable instructions 228 may cause the processor(s) 206 to provide commands or other control signals to the light source 208, the camera 212, the display 204, spectrometer 210, and/or other components of the computing device 218, such as the memory controller 220 (Paragraph 18).
Therefore taking the combined teachings of Shirai and Hu, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have an image capturing apparatus and a lighting device comprising at least one memory configured to store instructions, and at least one processor in communication with the at least one memory and configured to execute the instructions in order to easily change and implement the instructions automatically or manually based on the user’s needs and experimental goals thereby making the process more efficient.
[Claims 18-20]
These claims are method and computer readable storage claims corresponding to apparatus claims 15 and 3 and are therefore analyzed and rejected based upon apparatus claims 15 and 3 respectively.
Claim(s) 6 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Shirai et al. (US PGPUB 20120187275), Hu et al. (US PGPUB 20220067544) and in further view of Mizushiro (US PGPUB 20140210869).
[Claims 6 and 13]
Shirai in view of Hu fails to teach wherein the instructions are executed to further function as: a detection unit configured to detect luminance information from an image acquired by the image sensor, and a setting unit configured to set a duty ratio in the PWM control, based on the detected luminance information. However Mizushiro teaches a light source control section 13 generates the PWM signal with a predetermined duty ratio based on the required luminance supplied from the image processing section 11, and then supplies the light source 20 with the PWM signal to thereby perform the PWM control of the light source 20 (Paragraph 58).
Therefore taking the combined teachings of Shirai, Hu and Mizushiro, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have a detection unit configured to detect luminance information from an image acquired by the image sensor, and a setting unit configured to set a duty ratio in the PWM control, based on the detected luminance information in order for low resolution image difficult to be visually recognized by the user in the case of displaying the image using the light modulation device.
Claim(s) 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Shirai et al. (US PGPUB 20120187275), Hu et al. (US PGPUB 20220067544) and in further view of MICHENTHALER et al. (US PGPUB 20230125037).
[Claims 7 and 14]
Shirai in view of Hu fails to teach wherein the instructions are executed to further function as a warning unit configured to output a warning in a case where the second control unit does not support the frequency in the PWM control, which is determined based on the charge accumulation time. However Michenthaler teaches an ECU 120 expects to receive a “data not ready” warning in response to the first trigger after the PWM frequency increase and can discard the data frame accordingly (e.g., since the ECU 120 does not need the sensor data value at time Ts.sub.B/2 after the PWM frequency increase). Since the sensor 105 adapts the internal trigger level value, the sensor 105 expects another trigger at a time 2*Ts.sub.B/2 after the PWM frequency increase, and prepares and transmits the sensor data D2 accordingly (Paragraph 70).
Therefore taking the combined teachings of Shirai, Hu and Michenthaler, it would be obvious to one skilled in the art before the effective filing date of the invention to have been motivated to have output a warning in a case where the second control unit does not support the frequency in the PWM control, which is determined based on the charge accumulation time in order to change the frequency used by the system easily thereby preventing an error.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YOGESH K AGGARWAL whose telephone number is (571)272-7360. The examiner can normally be reached Monday - Friday 9:30-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sinh Tran can be reached at 5712727564. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YOGESH K AGGARWAL/Primary Examiner, Art Unit 2637
Prosecution Insights