DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-20 are pending for examination in the application filed 12/16/2025. Claims 1, 4, 7, 9, 12, 15, 17, and 20 have been amended.
Priority
Acknowledgement is made of Applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent application CN202311324462.4 filed on 10/12/2023.
Response to Arguments and Amendments
The 35 U.S.C. 112(b) rejections of claims 4, 7, 12, 15, and 20 are withdrawn in view of the amendments. The 35 U.S.C. 101 rejections of claims 1-2 are withdrawn in view of the amendments.
Applicant’s arguments with respect to claims 1-20 have been considered but are moot because the new ground of rejection does not rely on the combination of references applied in the prior rejection of record for any teaching or matter specifically challenged in the argument, as facilitated by the newly added amendments.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-2, 8-10, and 16-18 are rejected under 35 U.S.C. 103 as being unpatentable over Lee (CN116258999A) in view of Hanna (US20180365269A1) and Wang (CN105652217A).
Regarding claim 1, Lee teaches a method for testing indicator light comprising: receiving a surveillance video of an indicator light working on an electronic equipment ([pg. 3 para. 3] the purpose of the invention is to provide a video stream based on the indicator lamp state detecting method, can realize the multi-state monitoring (normally bright, normally extinguish; at the same time satisfy of the detection precision requirement. flicker frequency and colour change), which greatly expands the detection algorithm to the indication lamp state diversity monitoring range. The invention is used for monitoring the machine room power device operation state, mainly used for monitoring the operation state of the power device by the inspection robot);
determining a target location of the indicator lights in each frame image of the surveillance video according to a template image, and the template image comprising a location mark of the indicator light ([pg. 3 para. 7-9] An indicator light state detection method based on video stream, wherein based on the indication lamp panel image data under various states, forming a data set by marking the position of each indication lamp in the image, and then learning the marked data set by building improved Yolov4 deep learning model to obtain the indication lamp state detection model; using the trained model to detect the video stream to obtain the position area of each indicator light, and intercepting the position area of each indicator light as ROI area);
obtaining image information of the indicator light at the target location according to the location mark ([pg. 4 para 1] converting the ROI area of each indicator light from RGB color space to HSV color space, and performing channel separation in the HSV color space, respectively obtaining H and V channels; according to the difference of the pixel value in the V channel image under the two states of lighting and extinguishing the indicator lamp, by setting the threshold value of the pixel value of the V channel to judge the state of the indicator lamp; determining the colour of the indicator light according to the difference of the pixel value range of each colour in the H channel under the condition that the indicator light is lightened);
determining a flashing frequency of the indicator light according to the image information ([pg. 4 para 9] Further, in the process of detecting the indicating lamp state of the video stream, the frame rate requirement of the camera integrated on the inspection robot, the flicker frequency of the state indicating lamp on the device to satisfy detected, namely the camera frame rate is greater than the indicating lamp flicker frequency; and collecting the indicating lamp panel state video, according to the flicker frequency of each indicator lamp, the video duration is not less than 3 ~ 5 times of stroboscopic period);
determining a working status of the indicator light according to the flashing frequency ([pg. 5 para. 3] Further, indicating the lamp data management list of order statistic state change of each indicator lamp, to obtain the working state of each indicator lamp, comprising a normally bright, normal and flash and flash and slow flash flicker frequency).
Lee does not teach determining a flashing frequency and a duty cycle of the indicator light according to the image information, and comprising: obtaining brightness values at the target location in each video frame of the surveillance video, and the video frame corresponding to a timestamp; in a condition that a brightness value at the target location in the video frame at the timestamp is greater than a brightness value at the target location in the video frame at a last timestamp, and a brightness value difference of the video frame between the time stamp and the last time stamp is greater than a preset threshold, marking the video frame as a start frame; calculating a flashing period of the indicator light according to timestamps of any two adjacent start frames, determining the flashing frequency of the indicator light according to the flashing period; in a condition that a difference between the brightness value at the target location in the video frame at the time stamp and the brightness value at the target location in any start frame is less than or equal to the preset threshold, determining the video frame at the time stamp as a light frame; determining the duty cycle of the indicator light by calculating a ratio of a number of the light frames per unit time to a total number of video frames per unit times.
Hanna, in the same field of endeavor of analyzing an indicator light, teaches determining a flashing frequency and a duty cycle of the indicator light according to the image information ([0037] The registration areas, status indicator areas, reference imagery and acquired video is then passed to the Indicator Locations Identification module as shown in FIG. 3. [0042] The next step may be the indicator output estimation module illustratively shown in FIG. 3, and in more detail illustrated in FIG. 6. In this step the aligned illumination areas may be measured and characterized…The temporal status may be characterized by a duty cycle (mark to space ratio) which reflects the percentage of time that the device is on compared to off, and also the period, which reflects the frequency at which a pulsing pattern repeats),
and comprising: obtaining brightness values at the target location in each video frame of the surveillance video, and the video frame corresponding to a timestamp; in a condition that a brightness value at the target location in the video frame at the timestamp is greater than a brightness value at the target location in the video frame at a last timestamp, and a brightness value difference of the video frame between the time stamp and the last time stamp is greater than a preset threshold, marking the video frame as a start frame ([0043] The indicator output estimation module, illustratively shown in the middle of FIG. 7, may take each aligned indicator region and may extract a numerical value for the indicator, which may be binary, reflecting whether the LED is on or off, for example. This may be performed in some embodiments by detecting the brightness of the pixels in each indicator region, optionally with respect to each other or another region on the device (such as the reference areas) that are unchanged during status indicator changes in order to normalize for the effects of differences in ambient illumination over time. An example normalization algorithm is to detect the difference between the average intensity in the indicator region and the average intensity in an adjacent aligned reference region. If the difference exceeds a threshold, then the LED is deemed to be turned on. If the difference is below a threshold, then the LED is deemed to be turned off. [0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator. The duty cycle may be computed as the ratio of on time to off time for an indicator…the period of the pulsing may be computed as the time between off/on transitions of a status indicator. For example, the off/on transition may be detected on frames 6,12,19,25 of a sequence. The period calculated for each transition is then (12−6)=6, (19−12)=7, and (25−19)=6 respectively. This results in an average period of 6.33 frames);
calculating a flashing period of the indicator light according to timestamps of any two adjacent start frames, determining the flashing frequency of the indicator light according to the flashing period ([0045] In some embodiments and for a simple pulsing indicator, the period of the pulsing may be computed as the time between off/on transitions of a status indicator. For example, the off/on transition may be detected on frames 6,12,19,25 of a sequence. The period calculated for each transition is then (12−6)=6, (19−12)=7, and (25−19)=6 respectively. This results in an average period of 6.33 frames. [0042] The temporal status may be characterized by a duty cycle (mark to space ratio) which reflects the percentage of time that the device is on compared to off, and also the period, which reflects the frequency at which a pulsing pattern repeats);
in a condition that a difference between the brightness value at the target location in the video frame at the time stamp and the brightness value at the target location in any start frame is less than or equal to the preset threshold, determining the video frame at the time stamp as a light frame ([0043] The indicator output estimation module, illustratively shown in the middle of FIG. 7, may take each aligned indicator region and may extract a numerical value for the indicator, which may be binary, reflecting whether the LED is on or off, for example. This may be performed in some embodiments by detecting the brightness of the pixels in each indicator region, optionally with respect to each other. [0044] In the case of a color indication LED, then in some embodiments each of the R,G,B color responses may be measured and the threshold process repeated for each color. In some embodiments, the trainer may manually adjust the threshold until the detected status of a particular indicator light matches the actual status. [0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator);
determining the duty cycle of the indicator light by calculating a ratio of a number of the light frames per unit time to a total number of video frames per unit times ([0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator. The duty cycle may be computed as the ratio of on time to off time for an indicator. For example, if there are 100 frames acquired and in 65 of them the indicator is green and in 35 the indicator is off, then the duty cycle is computed to be 65%);
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee of determining the working status of an indicator light with the determining a duty cycle and comparing the brightness values at the target location of Hanna so that "The characterizations may then be passed into a reference characterization comparison module…A distance estimation module may compute a distance or difference vector between each set of reference status characteristics for each failure mode and the observed inspection status characteristics" [Hanna 0055].
Lee does not teach determining a working status of the indicator light according to the duty cycle.
Wang, in the same field of endeavor of indicator light testing, teaches determining a working status of the indicator light according to the duty cycle ([pg. 3 para. 1] In this embodiment, camera 400 shooting 30 photographs per second to the blinking frequency is 1 Hz of the alarm lamp, generally taking 4-6 seconds to ensure two complete twinkling period is collected. Correct sequence is from a certain time starting 30 continuously determines the alarm light of the picture, 30 continuously determines the alarm extinguishment of photo and repeating this sequence according to the shooting time. Under the condition of allowing the certain error, namely judging whether the flash logic of the alarm lamp frequency duty cycle of alarm lamp twinkling correctly and can be calculated).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee with the teachings of Wang to determine a working status of an indicator light based on the duty cycle because "compared with the existing technology with the naked eye, vehicle warning lamp flicker test method and system of the invention has accurate test result without error" [pg. 3 para. 2].
Regarding claim 2, Lee, Hanna, and Wang teach the method of claim 1. Lee further teaches wherein obtaining the template image comprises: obtaining a target image comprising the indicator light; locating a location of the indicator light in the target image, and obtaining the target location; recording the location mark in the target image by the target location, and obtaining the template image ([pg. 3 para. 7-9] An indicator light state detection method based on video stream, wherein based on the indication lamp panel image data under various states, forming a data set by marking the position of each indication lamp in the image).
Regarding claim 8, Lee, Hanna, and Wang teach the method of claim 1. Hanna teaches wherein the duty cycle is a ratio of a length of time that the indicator light is on per unit time in the surveillance video to a unit time ([0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator. The duty cycle may be computed as the ratio of on time to off time for an indicator. For example, if there are 100 frames acquired and in 65 of them the indicator is green and in 35 the indicator is off, then the duty cycle is computed to be 65%).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee with the teachings of Hanna to determine the duty cycle because "The characterizations may then be passed into a reference characterization comparison module…A distance estimation module may compute a distance or difference vector between each set of reference status characteristics for each failure mode and the observed inspection status characteristics…The distance vector computation may also involve the duty cycle and period of the illuminator status lights" [Hanna 0055].
Regarding claim 9, Lee teaches an electronic device comprising: a processor; and a non-transitory storage medium, coupled to the processor, that stores a plurality of instructions, which cause the processor to ([pg. 8 para. 4] The present invention is described with reference to a method according to an embodiment of the present invention, an apparatus (apparatus), and a computer program product. It should be understood that each flow can be implemented by computer program instructions. these computer program instructions can be provided to a general purpose computer and a special purpose computer, The processor of the embedded processor or other programmable data processing device generates a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate the device for realizing the function appointed in one process or multiple flows):
receive a surveillance video of an indicator light working on an electronic equipment ([pg. 3 para. 3] the purpose of the invention is to provide a video stream based on the indicator lamp state detecting method, can realize the multi-state monitoring (normally bright, normally extinguish; at the same time satisfy of the detection precision requirement. flicker frequency and colour change), which greatly expands the detection algorithm to the indication lamp state diversity monitoring range. The invention is used for monitoring the machine room power device operation state, mainly used for monitoring the operation state of the power device by the inspection robot);
determine a target location of the indicator lights in each frame image of the surveillance video according to a template image, and the template image comprising a location mark of the indicator light ([pg. 3 para. 7-9] An indicator light state detection method based on video stream, wherein based on the indication lamp panel image data under various states, forming a data set by marking the position of each indication lamp in the image, and then learning the marked data set by building improved Yolov4 deep learning model to obtain the indication lamp state detection model; using the trained model to detect the video stream to obtain the position area of each indicator light, and intercepting the position area of each indicator light as ROI area);
obtain image information of the indicator light at the target location according to the location mark ([pg. 4 para 1] converting the ROI area of each indicator light from RGB color space to HSV color space, and performing channel separation in the HSV color space, respectively obtaining H and V channels; according to the difference of the pixel value in the V channel image under the two states of lighting and extinguishing the indicator lamp, by setting the threshold value of the pixel value of the V channel to judge the state of the indicator lamp; determining the colour of the indicator light according to the difference of the pixel value range of each colour in the H channel under the condition that the indicator light is lightened);
determine a flashing frequency of the indicator light according to the image information ([pg. 4 para 9] Further, in the process of detecting the indicating lamp state of the video stream, the frame rate requirement of the camera integrated on the inspection robot, the flicker frequency of the state indicating lamp on the device to satisfy detected, namely the camera frame rate is greater than the indicating lamp flicker frequency; and collecting the indicating lamp panel state video, according to the flicker frequency of each indicator lamp, the video duration is not less than 3 ~ 5 times of stroboscopic period);
determine a working status of the indicator light according to the flashing frequency ([pg. 5 para. 3] Further, indicating the lamp data management list of order statistic state change of each indicator lamp, to obtain the working state of each indicator lamp, comprising a normally bright, normal and flash and flash and slow flash flicker frequency).
Lee does not teach determine a flashing frequency and a duty cycle of the indicator light according to the image information, and comprising: obtaining brightness values at the target location in each video frame of the surveillance video, and the video frame corresponding to a timestamp; in a condition that a brightness value at the target location in the video frame at the timestamp is greater than a brightness value at the target location in the video frame at a last timestamp, and a brightness value difference of the video frame between the time stamp and the last time stamp is greater than a preset threshold, marking the video frame as a start frame; calculating a flashing period of the indicator light according to timestamps of any two adjacent start frames, determining the flashing frequency of the indicator light according to the flashing period; in a condition that a difference between the brightness value at the target location in the video frame at the time stamp and the brightness value at the target location in any start frame is less than or equal to the preset threshold, determining the video frame at the time stamp as a light frame; determining the duty cycle of the indicator light by calculating a ratio of a number of the light frames per unit time to a total number of video frames per unit times.
Hanna, in the same field of endeavor of analyzing an indicator light, teaches determining a flashing frequency and a duty cycle of the indicator light according to the image information ([0037] The registration areas, status indicator areas, reference imagery and acquired video is then passed to the Indicator Locations Identification module as shown in FIG. 3. [0042] The next step may be the indicator output estimation module illustratively shown in FIG. 3, and in more detail illustrated in FIG. 6. In this step the aligned illumination areas may be measured and characterized…The temporal status may be characterized by a duty cycle (mark to space ratio) which reflects the percentage of time that the device is on compared to off, and also the period, which reflects the frequency at which a pulsing pattern repeats),
and comprising: obtaining brightness values at the target location in each video frame of the surveillance video, and the video frame corresponding to a timestamp; in a condition that a brightness value at the target location in the video frame at the timestamp is greater than a brightness value at the target location in the video frame at a last timestamp, and a brightness value difference of the video frame between the time stamp and the last time stamp is greater than a preset threshold, marking the video frame as a start frame ([0043] The indicator output estimation module, illustratively shown in the middle of FIG. 7, may take each aligned indicator region and may extract a numerical value for the indicator, which may be binary, reflecting whether the LED is on or off, for example. This may be performed in some embodiments by detecting the brightness of the pixels in each indicator region, optionally with respect to each other or another region on the device (such as the reference areas) that are unchanged during status indicator changes in order to normalize for the effects of differences in ambient illumination over time. An example normalization algorithm is to detect the difference between the average intensity in the indicator region and the average intensity in an adjacent aligned reference region. If the difference exceeds a threshold, then the LED is deemed to be turned on. If the difference is below a threshold, then the LED is deemed to be turned off. [0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator. The duty cycle may be computed as the ratio of on time to off time for an indicator…the period of the pulsing may be computed as the time between off/on transitions of a status indicator. For example, the off/on transition may be detected on frames 6,12,19,25 of a sequence. The period calculated for each transition is then (12−6)=6, (19−12)=7, and (25−19)=6 respectively. This results in an average period of 6.33 frames);
calculating a flashing period of the indicator light according to timestamps of any two adjacent start frames, determining the flashing frequency of the indicator light according to the flashing period ([0045] In some embodiments and for a simple pulsing indicator, the period of the pulsing may be computed as the time between off/on transitions of a status indicator. For example, the off/on transition may be detected on frames 6,12,19,25 of a sequence. The period calculated for each transition is then (12−6)=6, (19−12)=7, and (25−19)=6 respectively. This results in an average period of 6.33 frames. [0042] The temporal status may be characterized by a duty cycle (mark to space ratio) which reflects the percentage of time that the device is on compared to off, and also the period, which reflects the frequency at which a pulsing pattern repeats);
in a condition that a difference between the brightness value at the target location in the video frame at the time stamp and the brightness value at the target location in any start frame is less than or equal to the preset threshold, determining the video frame at the time stamp as a light frame ([0043] The indicator output estimation module, illustratively shown in the middle of FIG. 7, may take each aligned indicator region and may extract a numerical value for the indicator, which may be binary, reflecting whether the LED is on or off, for example. This may be performed in some embodiments by detecting the brightness of the pixels in each indicator region, optionally with respect to each other. [0044] In the case of a color indication LED, then in some embodiments each of the R,G,B color responses may be measured and the threshold process repeated for each color. In some embodiments, the trainer may manually adjust the threshold until the detected status of a particular indicator light matches the actual status. [0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator);
determining the duty cycle of the indicator light by calculating a ratio of a number of the light frames per unit time to a total number of video frames per unit times ([0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator. The duty cycle may be computed as the ratio of on time to off time for an indicator. For example, if there are 100 frames acquired and in 65 of them the indicator is green and in 35 the indicator is off, then the duty cycle is computed to be 65%);
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Lee of determining the working status of an indicator light with the determining a duty cycle and comparing the brightness values at the target location of Hanna so that "The characterizations may then be passed into a reference characterization comparison module…A distance estimation module may compute a distance or difference vector between each set of reference status characteristics for each failure mode and the observed inspection status characteristics" [Hanna 0055].
Lee does not teach determining a working status of the indicator light according to the duty cycle.
Wang, in the same field of endeavor of indicator light testing, teaches determining a working status of the indicator light according to the duty cycle ([pg. 3 para. 1] In this embodiment, camera 400 shooting 30 photographs per second to the blinking frequency is 1 Hz of the alarm lamp, generally taking 4-6 seconds to ensure two complete twinkling period is collected. Correct sequence is from a certain time starting 30 continuously determines the alarm light of the picture, 30 continuously determines the alarm extinguishment of photo and repeating this sequence according to the shooting time. Under the condition of allowing the certain error, namely judging whether the flash logic of the alarm lamp frequency duty cycle of alarm lamp twinkling correctly and can be calculated).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Lee with the teachings of Wang to determine a working status of an indicator light based on the duty cycle because "compared with the existing technology with the naked eye, vehicle warning lamp flicker test method and system of the invention has accurate test result without error" [pg. 3 para. 2].
Regarding claim 10, Lee, Hanna, and Wang teach the device of claim 9. Lee further teaches obtain a target image comprising the indicator light; locate a location of the indicator light in the target image, and obtain the target location; record the location mark in the target image by the target location, and obtain the template image ([pg. 3 para. 7-9] An indicator light state detection method based on video stream, wherein based on the indication lamp panel image data under various states, forming a data set by marking the position of each indication lamp in the image).
Regarding claim 16, Lee, Hanna, and Wang teach the device of claim 9. Wang teaches wherein the duty cycle is a ratio of a length of time that the indicator light is on per unit time in the surveillance video to a unit time ([pg. 1 para. 1 using camera shooting video in a state of twinkling alarm lamp, then the video shooting is transmitted to the image processing unit for
identifying and counting the each photo picture by the image processing unit continuously lighted and extinguished continuously photograph number. [pg. 3 para. 1] In this embodiment, camera 400 shooting 30 photographs per second to the blinking frequency is 1 Hz of the alarm lamp, generally taking 4-6 seconds to ensure two complete twinkling period is collected. Correct sequence is from a certain time starting 30 continuously determines the alarm light of the picture, 30 continuously determines the alarm extinguishment of photo and repeating this sequence according to the shooting time. Under the condition of allowing the certain error, namely judging whether the flash logic of the alarm lamp frequency duty cycle of alarm lamp twinkling correctly and can be calculated).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Lee with the teachings of Wang to determine a duty cycle because "compared with the existing technology with the naked eye, vehicle warning lamp flicker test method and system of the invention has accurate test result without error" [pg. 3 para. 2].
Regarding claim 17, Lee teaches a non-transitory storage medium having stored thereon instructions that, when executed by at least one processor of an electronic device, causes the at least one processor to perform a method for testing indicator light, the method comprising ([pg. 8 para. 4] The present invention is described with reference to a method according to an embodiment of the present invention, an apparatus (apparatus), and a computer program product. It should be understood that each flow can be implemented by computer program instructions. these computer program instructions can be provided to a general purpose computer and a special purpose computer, The processor of the embedded processor or other programmable data processing device generates a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate the device for realizing the function appointed in one process or multiple flows):
receiving a surveillance video of an indicator light working on an electronic equipment ([pg. 3 para. 3] the purpose of the invention is to provide a video stream based on the indicator lamp state detecting method, can realize the multi-state monitoring (normally bright, normally extinguish; at the same time satisfy of the detection precision requirement. flicker frequency and colour change), which greatly expands the detection algorithm to the indication lamp state diversity monitoring range. The invention is used for monitoring the machine room power device operation state, mainly used for monitoring the operation state of the power device by the inspection robot);
determining a target location of the indicator lights in each frame image of the surveillance video according to a template image, and the template image comprising a location mark of the indicator light ([pg. 3 para. 7-9] An indicator light state detection method based on video stream, wherein based on the indication lamp panel image data under various states, forming a data set by marking the position of each indication lamp in the image, and then learning the marked data set by building improved Yolov4 deep learning model to obtain the indication lamp state detection model; using the trained model to detect the video stream to obtain the position area of each indicator light, and intercepting the position area of each indicator light as ROI area);
obtaining image information of the indicator light at the target location according to the location mark ([pg. 4 para 1] converting the ROI area of each indicator light from RGB color space to HSV color space, and performing channel separation in the HSV color space, respectively obtaining H and V channels; according to the difference of the pixel value in the V channel image under the two states of lighting and extinguishing the indicator lamp, by setting the threshold value of the pixel value of the V channel to judge the state of the indicator lamp; determining the colour of the indicator light according to the difference of the pixel value range of each colour in the H channel under the condition that the indicator light is lightened);
determining a flashing frequency of the indicator light according to the image information ([pg. 4 para 9] Further, in the process of detecting the indicating lamp state of the video stream, the frame rate requirement of the camera integrated on the inspection robot, the flicker frequency of the state indicating lamp on the device to satisfy detected, namely the camera frame rate is greater than the indicating lamp flicker frequency; and collecting the indicating lamp panel state video, according to the flicker frequency of each indicator lamp, the video duration is not less than 3 ~ 5 times of stroboscopic period);
determining a working status of the indicator light according to the flashing frequency ([pg. 5 para. 3] Further, indicating the lamp data management list of order statistic state change of each indicator lamp, to obtain the working state of each indicator lamp, comprising a normally bright, normal and flash and flash and slow flash flicker frequency).
Lee does not teach determining a flashing frequency and a duty cycle of the indicator light according to the image information, and comprising: obtaining brightness values at the target location in each video frame of the surveillance video, and the video frame corresponding to a timestamp; in a condition that a brightness value at the target location in the video frame at the timestamp is greater than a brightness value at the target location in the video frame at a last timestamp, and a brightness value difference of the video frame between the time stamp and the last time stamp is greater than a preset threshold, marking the video frame as a start frame; calculating a flashing period of the indicator light according to timestamps of any two adjacent start frames, determining the flashing frequency of the indicator light according to the flashing period; in a condition that a difference between the brightness value at the target location in the video frame at the time stamp and the brightness value at the target location in any start frame is less than or equal to the preset threshold, determining the video frame at the time stamp as a light frame; determining the duty cycle of the indicator light by calculating a ratio of a number of the light frames per unit time to a total number of video frames per unit times.
Hanna, in the same field of endeavor of analyzing an indicator light, teaches determining a flashing frequency and a duty cycle of the indicator light according to the image information ([0037] The registration areas, status indicator areas, reference imagery and acquired video is then passed to the Indicator Locations Identification module as shown in FIG. 3. [0042] The next step may be the indicator output estimation module illustratively shown in FIG. 3, and in more detail illustrated in FIG. 6. In this step the aligned illumination areas may be measured and characterized…The temporal status may be characterized by a duty cycle (mark to space ratio) which reflects the percentage of time that the device is on compared to off, and also the period, which reflects the frequency at which a pulsing pattern repeats),
and comprising: obtaining brightness values at the target location in each video frame of the surveillance video, and the video frame corresponding to a timestamp; in a condition that a brightness value at the target location in the video frame at the timestamp is greater than a brightness value at the target location in the video frame at a last timestamp, and a brightness value difference of the video frame between the time stamp and the last time stamp is greater than a preset threshold, marking the video frame as a start frame ([0043] The indicator output estimation module, illustratively shown in the middle of FIG. 7, may take each aligned indicator region and may extract a numerical value for the indicator, which may be binary, reflecting whether the LED is on or off, for example. This may be performed in some embodiments by detecting the brightness of the pixels in each indicator region, optionally with respect to each other or another region on the device (such as the reference areas) that are unchanged during status indicator changes in order to normalize for the effects of differences in ambient illumination over time. An example normalization algorithm is to detect the difference between the average intensity in the indicator region and the average intensity in an adjacent aligned reference region. If the difference exceeds a threshold, then the LED is deemed to be turned on. If the difference is below a threshold, then the LED is deemed to be turned off. [0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator. The duty cycle may be computed as the ratio of on time to off time for an indicator…the period of the pulsing may be computed as the time between off/on transitions of a status indicator. For example, the off/on transition may be detected on frames 6,12,19,25 of a sequence. The period calculated for each transition is then (12−6)=6, (19−12)=7, and (25−19)=6 respectively. This results in an average period of 6.33 frames);
calculating a flashing period of the indicator light according to timestamps of any two adjacent start frames, determining the flashing frequency of the indicator light according to the flashing period ([0045] In some embodiments and for a simple pulsing indicator, the period of the pulsing may be computed as the time between off/on transitions of a status indicator. For example, the off/on transition may be detected on frames 6,12,19,25 of a sequence. The period calculated for each transition is then (12−6)=6, (19−12)=7, and (25−19)=6 respectively. This results in an average period of 6.33 frames. [0042] The temporal status may be characterized by a duty cycle (mark to space ratio) which reflects the percentage of time that the device is on compared to off, and also the period, which reflects the frequency at which a pulsing pattern repeats);
in a condition that a difference between the brightness value at the target location in the video frame at the time stamp and the brightness value at the target location in any start frame is less than or equal to the preset threshold, determining the video frame at the time stamp as a light frame ([0043] The indicator output estimation module, illustratively shown in the middle of FIG. 7, may take each aligned indicator region and may extract a numerical value for the indicator, which may be binary, reflecting whether the LED is on or off, for example. This may be performed in some embodiments by detecting the brightness of the pixels in each indicator region, optionally with respect to each other. [0044] In the case of a color indication LED, then in some embodiments each of the R,G,B color responses may be measured and the threshold process repeated for each color. In some embodiments, the trainer may manually adjust the threshold until the detected status of a particular indicator light matches the actual status. [0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator);
determining the duty cycle of the indicator light by calculating a ratio of a number of the light frames per unit time to a total number of video frames per unit times ([0045] These feature-extracted results may then be passed to one embodiment of an Indicator Characterization Module illustratively shown at the bottom right of FIG. 7. This may analyze the extracted features over a time sequence for each indicator. The duty cycle may be computed as the ratio of on time to off time for an indicator. For example, if there are 100 frames acquired and in 65 of them the indicator is green and in 35 the indicator is off, then the duty cycle is computed to be 65%);
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the medium of Lee of determining the working status of an indicator light with the determining a duty cycle and comparing the brightness values at the target location of Hanna so that "The characterizations may then be passed into a reference characterization comparison module…A distance estimation module may compute a distance or difference vector between each set of reference status characteristics for each failure mode and the observed inspection status characteristics" [Hanna 0055].
Lee does not teach determining a working status of the indicator light according to the duty cycle.
Wang, in the same field of endeavor of indicator light testing, teaches determining a working status of the indicator light according to the duty cycle ([pg. 3 para. 1] In this embodiment, camera 400 shooting 30 photographs per second to the blinking frequency is 1 Hz of the alarm lamp, generally taking 4-6 seconds to ensure two complete twinkling period is collected. Correct sequence is from a certain time starting 30 continuously determines the alarm light of the picture, 30 continuously determines the alarm extinguishment of photo and repeating this sequence according to the shooting time. Under the condition of allowing the certain error, namely judging whether the flash logic of the alarm lamp frequency duty cycle of alarm lamp twinkling correctly and can be calculated).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the medium of Lee with the teachings of Wang to determine a working status of an indicator light based on the duty cycle because "compared with the existing technology with the naked eye, vehicle warning lamp flicker test method and system of the invention has accurate test result without error" [pg. 3 para. 2].
Regarding claim 18, Lee, Hanna, and Wang teach the medium of claim 17. Lee further teaches wherein obtaining the template image comprises: obtaining a target image comprising the indicator light; locating a location of the indicator light in the target image, and obtaining the target location; recording the location mark in the target image by the target location, and obtaining the template image ([pg. 3 para. 7-9] An indicator light state detection method based on video stream, wherein based on the indication lamp panel image data under various states, forming a data set by marking the position of each indication lamp in the image).
Claims 3-4, 11-12, and 19-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Hanna, Wang, and Shen (CN111259892B).
Regarding claim 3, Lee, Hanna, and Wang teach the method of claim 2. Lee further teaches wherein locating the location of the indicator light in the target image, and obtaining the target location comprises: determining any frame image adjacent to the target image in the surveillance video as a test image; determining all pixels in the target image as first pixels (first state), and determining all pixels in the test image as second pixels (second state); determining a brightness value of each of the first pixels, and determining a brightness value of each of the second pixels; calculating a brightness difference between one first pixel and one second pixel with the same coordinates ([pg. 7 para. 11] Step S71: processing the image pixel value of the V channel image extracted from the HSV colour space, because the pixel value range corresponding to the lighting and extinguishing two states of the indicating lamp in the V channel image has a large difference, so it can judge the indicating lamp state by setting the pixel value threshold value for the V channel image).
Lee does not teach in response that the brightness difference is greater than a preset threshold, determining coordinates of the first pixels corresponding to the brightness difference in the target image to be the target location.
Shen, in the same field of endeavor of indicator light detection, teaches in response that the brightness difference is greater than a preset threshold, determining coordinates of the first pixels corresponding to the brightness difference in the target image to be the target location ([pg. 7 para. 2-4] S31, using the Scharr operator to calculate the gradient image of the effective area in the template image, then according to the image pyramid of the gradient image, obtaining the multi-scale template gradient image; S32, calculating the gradient image of the whole image of the inspection image by using the Scharr operator, obtaining the inspection gradient image; S33. in the inspection gradient image according to the order (such as from left to right, from up to down), sequentially the standard correlation coefficient between the inspection gradient and the multi-scale template gradient image in the rectangular window M * N of the effective area size, obtaining the position corresponding to the highest score as the best matching position, the scale corresponding to the highest score is the best matching scale).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee with the teachings of Shen to determine the target location based on the brightness difference because "comparing the modeling data and the inspection data, obtaining the inspection result, greatly improv[es] the accuracy of the identification" [pg. 4 para. 3].
Regarding claim 4, Lee, Hanna, and Wang teach the method of claim 2. Lee further teaches wherein locating the location of the indicator light in the target image, and obtaining the target location comprises: determining a pixel value of each of first pixels in the target image ([pg. 4 para. 12] Step S71: processing the image pixel value of the V channel image extracted from the HSV colour Space).
Lee does not teach in response that the pixel value is within a preset pixel value range, determining coordinates of the first pixels corresponding to the pixel value in the target image to be the target location.
Shen teaches in response that the pixel value is within a preset pixel value range, determining coordinates of the first pixels corresponding to the pixel value in the target image to be the target location ([pg. 6 para. 5-6] S12, converting each image into the HSV colour space from the RGB colour space; S13, in the HSV colour space, calculating the mask graph of each image, for each pixel point, the value of the HSV range in the mask image of the indicator lamp colour HSV range is 1, otherwise is 0).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee with the teachings of Shen to determine the target location based on the pixel values because "comparing the modeling data and the inspection data, obtaining the inspection result, greatly improv[es] the accuracy of the identification" [pg. 4 para. 3].
Regarding claim 11, Lee, Hanna, and Wang teach the device of claim 10. Lee further teaches determine any frame image adjacent to the target image in the surveillance video as a test image; determine all pixels in the target image as first pixels (first state), and determine all pixels in the test image as second pixels (second state); determine a brightness value of each of the first pixels, and determine a brightness value of each of the second pixels; calculate a brightness difference between one first pixel and one second pixel with the same coordinates ([pg. 7 para. 11] Step S71: processing the image pixel value of the V channel image extracted from the HSV colour space, because the pixel value range corresponding to the lighting and extinguishing two states of the indicating lamp in the V channel image has a large difference, so it can judge the indicating lamp state by setting the pixel value threshold value for the V channel image).
Lee does not teach in response that the brightness difference is greater than a preset threshold, determine coordinates of the first pixels corresponding to the brightness difference in the target image to be the target location.
Shen, in the same field of endeavor of indicator light detection, teaches in response that the brightness difference is greater than a preset threshold, determine coordinates of the first pixels corresponding to the brightness difference in the target image to be the target location ([pg. 7 para. 2-4] S31, using the Scharr operator to calculate the gradient image of the effective area in the template image, then according to the image pyramid of the gradient image, obtaining the multi-scale template gradient image; S32, calculating the gradient image of the whole image of the inspection image by using the Scharr operator, obtaining the inspection gradient image; S33. in the inspection gradient image according to the order (such as from left to right, from up to down), sequentially the standard correlation coefficient between the inspection gradient and the multi-scale template gradient image in the rectangular window M * N of the effective area size, obtaining the position corresponding to the highest score as the best matching position, the scale corresponding to the highest score is the best matching scale).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Lee with the teachings of Shen to determine the target location based on the brightness difference because "comparing the modeling data and the inspection data, obtaining the inspection result, greatly improv[es] the accuracy of the identification" [pg. 4 para. 3].
Regarding claim 12, Lee, Hanna, and Wang teach the device of claim 10. Lee further teaches determine a pixel value of each of first pixels in the target image ([pg. 4 para. 12] Step S71: processing the image pixel value of the V channel image extracted from the HSV colour Space).
Lee does not teach in response that the pixel value is within a preset pixel value range, determine coordinates of the first pixels corresponding to the pixel value in the target image to be the target location.
Shen teaches in response that the pixel value is within a preset pixel value range, determine coordinates of the first pixels corresponding to the pixel value in the target image to be the target location ([pg. 6 para. 5-6] S12, converting each image into the HSV colour space from the RGB colour space; S13, in the HSV colour space, calculating the mask graph of each image, for each pixel point, the value of the HSV range in the mask image of the indicator lamp colour HSV range is 1, otherwise is 0).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Lee with the teachings of Shen to determine the target location based on the pixel values because "comparing the modeling data and the inspection data, obtaining the inspection result, greatly improv[es] the accuracy of the identification" [pg. 4 para. 3].
Regarding claim 19, Lee, Hanna, and Wang teach the medium of claim 18. Lee further teaches wherein locating the location of the indicator light in the target image, and obtaining the target location comprises: determining any frame image adjacent to the target image in the surveillance video as a test image; determining all pixels in the target image as first pixels (first state), and determining all pixels in the test image as second pixels (second state); determining a brightness value of each of the first pixels, and determining a brightness value of each of the second pixels; calculating a brightness difference between one first pixel and one second pixel with the same coordinates ([pg. 7 para. 11] Step S71: processing the image pixel value of the V channel image extracted from the HSV colour space, because the pixel value range corresponding to the lighting and extinguishing two states of the indicating lamp in the V channel image has a large difference, so it can judge the indicating lamp state by setting the pixel value threshold value for the V channel image).
Lee does not teach in response that the brightness difference is greater than a preset threshold, determining coordinates of the first pixels corresponding to the brightness difference in the target image to be the target location.
Shen, in the same field of endeavor of indicator light detection, teaches in response that the brightness difference is greater than a preset threshold, determining coordinates of the first pixels corresponding to the brightness difference in the target image to be the target location ([pg. 7 para. 2-4] S31, using the Scharr operator to calculate the gradient image of the effective area in the template image, then according to the image pyramid of the gradient image, obtaining the multi-scale template gradient image; S32, calculating the gradient image of the whole image of the inspection image by using the Scharr operator, obtaining the inspection gradient image; S33. in the inspection gradient image according to the order (such as from left to right, from up to down), sequentially the standard correlation coefficient between the inspection gradient and the multi-scale template gradient image in the rectangular window M * N of the effective area size, obtaining the position corresponding to the highest score as the best matching position, the scale corresponding to the highest score is the best matching scale).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the medium of Lee with the teachings of Shen to determine the target location based on the brightness difference because "comparing the modeling data and the inspection data, obtaining the inspection result, greatly improv[es] the accuracy of the identification" [pg. 4 para. 3].
Regarding claim 20, Lee, Hanna, and Wang teach the medium of claim 18. Lee further teaches wherein locating the location of the indicator light in the target image, and obtaining the target location comprises: determining a pixel value of each of first pixels in the target image ([pg. 4 para. 12] Step S71: processing the image pixel value of the V channel image extracted from the HSV colour Space).
Lee does not teach in response that the pixel value is within a preset pixel value range, determining coordinates of the first pixels corresponding to the pixel value in the target image to be the target location.
Shen teaches in response that the pixel value is within a preset pixel value range, determining coordinates of the first pixels corresponding to the pixel value in the target image to be the target location ([pg. 6 para. 5-6] S12, converting each image into the HSV colour space from the RGB colour space; S13, in the HSV colour space, calculating the mask graph of each image, for each pixel point, the value of the HSV range in the mask image of the indicator lamp colour HSV range is 1, otherwise is 0).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the medium of Lee with the teachings of Shen to determine the target location based on the pixel values because "comparing the modeling data and the inspection data, obtaining the inspection result, greatly improv[es] the accuracy of the identification" [pg. 4 para. 3].
Claims 5-6 and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Hanna, Wang, and Wu (US20140071281A1).
Regarding claim 5, Lee, Hanna, and Wang teach the method of claim 2. Lee further teaches sending the template image comprising the location mark to a terminal communicatively connected to an electronic device ([pg. 4 para. 5] step S12: marking indicator lamp panel data set, marking the position area of each indicator lamp in the image according to the sequence of the database management list, as the ROI area of the indicator lamp, and storing into the training data file).
Lee does not teach receiving location calibration confirmation information sent by the terminal; in response that the location calibration confirmation information is that a calibration is passed, receiving a monitoring video of the indicator light when the indicator light is working on the electronic equipment.
Wu, in the same field of endeavor of indicator light monitoring, teaches receiving location calibration confirmation information sent by the terminal; in response that the location calibration confirmation information is that a calibration is passed, receiving a monitoring video of the indicator light when the indicator light is working on the electronic equipment ([0002] Generally, this type of camera selectively captures video of a scene that is being surveyed. These cameras may then transmit image data to a remote server device for further processing, or may perform some or all of the processing on-board. [0014] FIG. 1 provides an overview of the method 10. The system starts a diagnostic routine at S12. At S14, a scene analysis is performed on data that is extracted from an image frame of the scene that is captured by the camera. The scene analysis is used to determine a severity (i.e., a probability and/or likelihood) of the camera fault P.sub.S being tested, such as, in one example, the camera misalignment and/or displacement. The subscript S is used to denote that the camera fault P.sub.S is determined from scene analysis. The camera fault P.sub.S is compared to first and second thresholds .eta..sub.1 and .eta..sub.2 at S16, S18, whereby the second threshold .eta..sub.2 is significantly larger than the first threshold .eta..sub.1. For the fault P.sub.S not exceeding the first threshold (P.sub.S.ltoreq..eta..sub.1) (YES at S16), no further action is required (S20). [0035] The reference object can include a stationary object and, more specifically, a stationary and permanent object, within the camera field of view. Non-limiting examples of objects can include a traffic indicator, such as a traffic light).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee with the teachings of Wu to perform location calibration "Because the image data is often used for fee collection and traffic enforcement, the ability of the camera to accurately render an image is essential to the performance of a traffic monitoring system" [Wu 0002].
Regarding claim 6, Lee, Hanna, and Wang, and Wu teach the method of claim 5. Wu teaches in response that the location calibration confirmation information indicates that the calibration is failed, repositioning the indicator light in the target image ([0048] Continuing with FIG. 3, a maintenance crew may need to reposition the camera for a larger displacement or for instances when calibration cannot remedy the problem, such as, for example, when the misalignment is severe, when faults are caused by near-field blockage, during out-of-focus conditions, and/or when there is illuminator failure, etc. A displacement exceeding the threshold value T.sub.2 (YES at S320) confirms that the camera has incurred a serious fault and that the camera is in need of on-site repair, maintenance, and/or replacement).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee with the teachings of Wu to perform location calibration "Because the image data is often used for fee collection and traffic enforcement, the ability of the camera to accurately render an image is essential to the performance of a traffic monitoring system" [Wu 0002].
Regarding claim 13, Lee, Hanna, and Wang teach the device of claim 10. Lee further teaches sending the template image comprising the location mark to a terminal communicatively connected to an electronic device ([pg. 4 para. 5] step S12: marking indicator lamp panel data set, marking the position area of each indicator lamp in the image according to the sequence of the database management list, as the ROI area of the indicator lamp, and storing into the training data file).
Lee does not teach receive location calibration confirmation information sent by the terminal; in response that the location calibration confirmation information is that a calibration is passed, receive a monitoring video of the indicator light when the indicator light is working on the electronic equipment
Wu, in the same field of endeavor of indicator light monitoring, teaches receive location calibration confirmation information sent by the terminal; in response that the location calibration confirmation information is that a calibration is passed, receive a monitoring video of the indicator light when the indicator light is working on the electronic equipment ([0002] Generally, this type of camera selectively captures video of a scene that is being surveyed. These cameras may then transmit image data to a remote server device for further processing, or may perform some or all of the processing on-board. [0014] FIG. 1 provides an overview of the method 10. The system starts a diagnostic routine at S12. At S14, a scene analysis is performed on data that is extracted from an image frame of the scene that is captured by the camera. The scene analysis is used to determine a severity (i.e., a probability and/or likelihood) of the camera fault P.sub.S being tested, such as, in one example, the camera misalignment and/or displacement. The subscript S is used to denote that the camera fault P.sub.S is determined from scene analysis. The camera fault P.sub.S is compared to first and second thresholds .eta..sub.1 and .eta..sub.2 at S16, S18, whereby the second threshold .eta..sub.2 is significantly larger than the first threshold .eta..sub.1. For the fault P.sub.S not exceeding the first threshold (P.sub.S.ltoreq..eta..sub.1) (YES at S16), no further action is required (S20). [0035] The reference object can include a stationary object and, more specifically, a stationary and permanent object, within the camera field of view. Non-limiting examples of objects can include a traffic indicator, such as a traffic light).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Lee with the teachings of Wu to perform location calibration "Because the image data is often used for fee collection and traffic enforcement, the ability of the camera to accurately render an image is essential to the performance of a traffic monitoring system" [Wu 0002].
Regarding claim 14, Lee, Hanna, Wang, and Wu teach the device of claim 13. Wu teaches in response that the location calibration confirmation information indicates that the calibration is failed, reposition the indicator light in the target image ([0048] Continuing with FIG. 3, a maintenance crew may need to reposition the camera for a larger displacement or for instances when calibration cannot remedy the problem, such as, for example, when the misalignment is severe, when faults are caused by near-field blockage, during out-of-focus conditions, and/or when there is illuminator failure, etc. A displacement exceeding the threshold value T.sub.2 (YES at S320) confirms that the camera has incurred a serious fault and that the camera is in need of on-site repair, maintenance, and/or replacement).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Lee with the teachings of Wu to perform location calibration "Because the image data is often used for fee collection and traffic enforcement, the ability of the camera to accurately render an image is essential to the performance of a traffic monitoring system" [Wu 0002].
Claims 7 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Lee in view of Hanna, Wang, Wu, and Chen (WO2018113111A1).
Regarding claim 7, Lee, Hanna, Wang, and Wu teach the method of claim 6. Lee further teaches wherein determining the working status of the indicator light according to the flashing frequency, comprises: in response that the flashing frequency satisfies a first condition, determining that the indicator light is in a normal working status ([pg. 5 para. 3] Further, indicating the lamp data management list of order statistic state change of each indicator lamp, to obtain the working state of each indicator lamp, comprising a normally bright, normal and flash and flash and slow flash flicker frequency).
Lee does not teach determining the working status of the indicator light according to the duty cycle.
Wang teaches determining the working status of the indicator light according to the duty cycle ([pg. 3 para. 1] In this embodiment, camera 400 shooting 30 photographs per second to the blinking frequency is 1 Hz of the alarm lamp, generally taking 4-6 seconds to ensure two complete twinkling period is collected. Correct sequence is from a certain time starting 30 continuously determines the alarm light of the picture, 30 continuously determines the alarm extinguishment of photo and repeating this sequence according to the shooting time. Under the condition of allowing the certain error, namely judging whether the flash logic of the alarm lamp frequency duty cycle of alarm lamp twinkling correctly and can be calculated).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee with the teachings of Wang to determine a working status of an indicator light based on the duty cycle because "compared with the existing technology with the naked eye, vehicle warning lamp flicker test method and system of the invention has accurate test result without error" [pg. 3 para. 2].
Lee does not teach the duty cycle satisfies a second condition, and the first condition indicating that an absolute error between the flashing frequency and a preset flashing frequency does not exceed a preset first difference, the second condition indicating that an absolute error between the duty cycle and a preset duty cycle does not exceed a preset second difference.
Chen, in the same field of endeavor of light emission analysis, teaches the duty cycle satisfies a second condition, and the first condition indicating that an absolute error between the flashing frequency and a preset flashing frequency does not exceed a preset first difference, the second condition indicating that an absolute error between the duty cycle and a preset duty cycle does not exceed a preset second difference ([pg. 12 para. 1] When the protective case is in the closed state, the image formed by the first light intensity value received by the light sensor is as shown by waveform 902…When the frequency of the waveform 902 is equal to the frequency of the square wave 901 or the difference is less than a preset threshold (such as 1 Hz, etc.), the first light intensity value received by the light sensor is obtained. The law of change is consistent with the first law. In a third aspect, the terminal can calculate the duty ratio of the waveform 902 and the duty ratio of the square wave 901. When the difference between the duty ratio of the waveform 902 and the duty ratio of the square wave 901 is less than a preset threshold, such as 0.05, then the light The variation law of the first light intensity value received by the sensor is consistent with the first law).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the method of Lee with the teachings of Chen to ensure the flashing frequency and duty cycle do not exceed a preset difference because "when the protective shell is in an open state, the terminal performs switching control on the light emitting device by using the first rule, and detects a change rule of the first light intensity value received by the light sensor. When the first rule is consistent, the terminal detects that the protective shell is closed, and the terminal can perform a first operation, such as a sleep operation" [pg. 2 para. 10].
Regarding claim 15, Lee, Hanna, Wang, and Wu teach the device of claim 14. Lee further teaches in response that the flashing frequency satisfies a first condition, determining that the indicator light is in a normal working status ([pg. 5 para. 3] Further, indicating the lamp data management list of order statistic state change of each indicator lamp, to obtain the working state of each indicator lamp, comprising a normally bright, normal and flash and flash and slow flash flicker frequency).
Lee does not teach the duty cycle satisfies a second condition, and the first condition indicating that an absolute error between the flashing frequency and a preset flashing frequency does not exceed a preset first difference, the second condition indicating that an absolute error between the duty cycle and a preset duty cycle does not exceed a preset second difference.
Chen, in the same field of endeavor of light emission analysis, teaches the duty cycle satisfies a second condition, and the first condition indicating that an absolute error between the flashing frequency and a preset flashing frequency does not exceed a preset first difference, the second condition indicating that an absolute error between the duty cycle and a preset duty cycle does not exceed a preset second difference ([pg. 12 para. 1] When the protective case is in the closed state, the image formed by the first light intensity value received by the light sensor is as shown by waveform 902…When the frequency of the waveform 902 is equal to the frequency of the square wave 901 or the difference is less than a preset threshold (such as 1 Hz, etc.), the first light intensity value received by the light sensor is obtained. The law of change is consistent with the first law. In a third aspect, the terminal can calculate the duty ratio of the waveform 902 and the duty ratio of the square wave 901. When the difference between the duty ratio of the waveform 902 and the duty ratio of the square wave 901 is less than a preset threshold, such as 0.05, then the light The variation law of the first light intensity value received by the sensor is consistent with the first law).
Therefore, it would have been obvious to a person of ordinary skill in the art at the time that the invention was made to modify the device of Lee with the teachings of Chen to ensure the flashing frequency and duty cycle do not exceed a difference because "when the protective shell is in an open state, the terminal performs switching control on the light emitting device by using the first rule, and detects a change rule of the first light intensity value received by the light sensor. When the first rule is consistent, the terminal detects that the protective shell is closed, and the terminal can perform a first operation, such as a sleep operation" [pg. 2 para. 10].
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jacqueline R Zak whose telephone number is (571)272-4077. The examiner can normally be reached M-F 9-5. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571) 270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JACQUELINE R ZAK/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666
Prosecution Insights