Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This office action is in response to application filed on January 9, 2025. The Preliminary Amendment filed on January 9, 2025 has been entered. Claims 1-12 and 14-21 are currently pending in the application.
Drawings
The drawings filed on January 9, 2025 are acknowledged and are acceptable.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-9 and 14-21 are rejected under 35 U.S.C. 103 as being unpatentable over Yang et al. (Chinese Publication No. CN112580600A; machine translation obtained from Espacenet; hereinafter as “Yang”) in view of Ji et al. (Chinese Publication No. CN113378667A; machine translation obtained from Espacenet; hereinafter as “Ji”).
As per claim 1, Yang discloses A warning method for environment detection (discloses a dust concentration detection method), comprising:
acquiring a video stream of a preset collection region (e.g., para. [0031]: “The dust concentration detection method provided in this application can be applied to the video surveillance system shown in Figure 1…” and “each camera device 160 can be set up at different positions on the ship loader according to the dust monitoring requirements…”), sequentially collecting frame images from the video stream as detection images (e.g., para. [0034]: “the server can collect operation videos of the bulk cargo terminal under different scenarios during the day and at night, and extract video frames from the operation videos. These video frames are then used as dust detection images to obtain the corresponding dust concentration.”), and acquiring an object detection result by inputting the detection images into an image recognition model (e.g., para. [0007]-[0008], [0035]: “Input the dust detection image into the pre-trained dust annotation model and obtain the target detection box output by the dust annotation model; the target detection box is used to annotate the dust location”; The disclosed YOLOv3-SPP dust-labeling model is therefore an image-recognition/detection model), wherein the object detection result comprises fugitive dust level information (e.g., abstract; para. [0057]: “obtaining a target detection frame output by the dust labeling model, wherein the target detection frame is used for marking a dust position; and calculating the target image transmissivity in the target detection frame in the dust detection image, and determining the dust concentration according to the target image transmissivity.”);
determining, in a case that the object detection result comprises a fugitive dust detection box, fugitive dust state information based on positioning information of the fugitive dust detection box (see e.g., para. [0036]: “The target detection bounding box is used to mark the location of dust in the dust detection image, and the bounded area shows the dust area in the image”);
using the fugitive dust state information and the fugitive dust level information as detection data (see e.g., para. [0057]-[0058]: A mapping relationship of image transmittance to dust concentration is obtained, and a dust concentration corresponding to the target image transmittance is determined based on the mapping relationship. The dust concentration can be divided into several levels); and
giving a fugitive dust warning in a case that a fugitive dust warning condition is satisfied based on the detection data and a historical detection data set (e.g., para. [0104]: “when the dust concentration is greater than a concentration threshold, confirming the dust detection image as a target detection image, and issuing an alarm when the number of target detection images is greater than an alarm threshold.”); and updating, in the case that the fugitive dust warning condition is satisfied based on the detection data and the historical detection data set, the historical detection data set based on the detection data (e.g., para. [0079]: “the alarm threshold and/or concentration threshold of the image can be adjusted according to the detection index requirements…”; Yang discloses adjustment of thresholds according to detection requirements. It does not disclose updating the dust alarm conditions based on the current detection data and a historical detection data set after determining that the alarm conditions are met), and returning to perform a step of sequentially collecting frame images from the video stream as detection images (Yang describes processing individual dust detection images/video frames, but does not disclose after updating the alarm conditions, the method returns to sequentially collecting additional frames from the video stream).
Yang discloses an environment/dust detection and alarm method employing an image recognition model, such as YOLO model, to detect fugitive-dust detection frame, and further discloses determining a dust level and generating an alarm based on dust detected in a plurality of images. Yang therefore provides the primary framework for detecting fugitive dust and generating a dust alarm. Yang does not teach, at least expressly, the combination of a video stream of a preset collection region with sequential frame collection, determining alarm conditions using the detection data and a historical detection data set, updating the alarm conditions based on the detection data and historical detection data set, and returning to sequential frame collection after the update.
However, in the same field of automated construction-site environmental monitoring using image-processing and machine-learning technique, Ji teaches acquiring images of a construction-site scene and detecting construction-site fugitive dust, examining the same image and/or temporally adjacent frames to continue detection (see e.g., abstract, para. [0010]-[0011], [0025]). Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to apply the sequential-image processing of Ji to the dust-monitoring system of Yang so as to provide continued monitoring of the construction-site area and thereby improve the continuity and reliability of dust detection.
Ji further teaches using image-processing/neural-network techniques to identify construction-site dust and soil and determining the position and category of the detected regions (see e.g., para. [0072], [0087]). Accordingly, incorporating such image-processing operations into the system of Yang would have provided a predictable implementation of the image acquisition and detection operations, while retaining Yang’s dust detection and alarm functionality.
With respect to the historical detection data, Yang already teaches determining an alarm based on dust detected in a plurality of images. In view of Ji’s teaching of processing temporally adjacent images, it would have been obvious to maintain detection information from previously processed images and use the current and prior detection results in determining whether a continuing dust condition satisfies an alarm criterion. Such an arrangement would permit the system to make the alarm determination from a temporal sequence of detection results rather than from an isolated image. Further, where Yang determines a dust level from the detected fugitive-dust information, it would have been obvious to use the output of the image recognition model as the dust information used for determining the alarm condition and corresponding alarm level. Ji reinforces this approach by expressly using image-recognition results to identify and classify construction-site dust and to generate warning information based on those detection results (see e.g., para. [0012]).
Thus, the combination of Yang and Ji would have provided a system that sequentially acquires images from a construction-site video stream, applies an image recognition model to the images to obtain fugitive-dust detection information, maintains detection information associated with successive images, and determines whether the accumulated detection results satisfy an alarm condition, thereby providing continued monitoring of the construction-site scene and more reliable identification of dust-related conditions. The respective teachings would have performed their established functions in the combination and would have produced a predictable result. Accordingly, the subject matter of claim 1 would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
As per claim 2, claim 1 is incorporated and Yang in view of Ji teaches: wherein the fugitive dust state information comprises a first state value representing a presence of fugitive dust and a second state value representing an absence of the fugitive dust; the historical detection data set comprises historical detection data corresponding to at least one frame of historical detection image collected historically; and
said determining the fugitive dust state information based on the positioning information of the fugitive dust detection box comprises:
determining whether the fugitive dust detection box satisfies a first preset condition based on the positioning information of the fugitive dust detection box;
determining that the fugitive dust state information is the first state value in a case that the fugitive dust detection box satisfies the first preset condition; and
determining that the fugitive dust state information is the second state value in a case that the fugitive dust detection box does not satisfy the first preset condition (see Yang, e.g., para. [0036]: “The target detection bounding box is used to mark the location of dust in the dust detection image, and the bounded area shows the dust area in the image”; para. [0104]: “when the dust concentration is greater than a concentration threshold, confirming the dust detection image as a target detection image, and issuing an alarm when the number of target detection images is greater than an alarm threshold.”; also see Ji, abstract; para. [0010]-[0011], [0025], [0087]: discloses determining whether dust is present from image recognition; processing temporally adjacent images; location/category-based detection).
A person of ordinary skill in the art before the effective filing date of the claimed invention would have represented the result of each frame’s dust detection as a dual detection state (i.e., dust present or dust absent) and retained the results of successive frames so that the temporal detection results could be evaluated against the alarm criterion already disclosed by Yang. Assigning numerical values to the two states and accumulating those values provides a straightforward implementation of determining whether a predetermined number of frames satisfy the dust condition.
As per claim 3, claim 1 is incorporated and Yang in view of Ji teaches: wherein the historical detection data set is capable of containing no more than a preset number of historical detection data items; and
said updating the historical detection data set based on the detection data comprises:
removing, in a case that a number of the historical detection data items in the historical detection data set is equal to the preset number, a historical detection data item with earliest storage time from a current historical detection data set, and adding the detection data to the historical detection data set as a new historical detection data item (see Yang, e.g., para. [0079]: “the alarm threshold and/or concentration threshold of the image can be adjusted according to the detection index requirements…”; also see Ji, e.g., para. [0061]: “at least one frame of the image to be detected that is temporally adjacent to the image to be detected is acquired. For example, the previous frame of the image to be detected or the next frame of the image to be detected.”).
Because the system of Yang, as supplemented by Ji, processes a sequence of detection images and uses detection results from previously processed images, it would have been desirable to limit the amount of retained historical detection information to a predetermined number of entries. When the system reached its set limit, it could remove the oldest detection result before adding the newest one. This would create a standard sliding window that kept only the most recent results. It would preserve the information needed to determine alarms, prevent the stored history from growing endlessly, and make the system’s behavior predictable.
As per claim 4, claim 1 is incorporated and Yang in view of Ji teaches: the method further comprising: using, in a case that the fugitive dust warning condition is not satisfied based on the historical detection data set and the fugitive dust warning condition is satisfied based on the detection data and the historical detection data set, a time on the condition that the detection images are collected as a fugitive dust start time, and generating fugitive dust warning information (see Yang, e.g., para. [0104]: “issuing an alarm when the number of target detection images is greater than an alarm threshold.”; para. [0078]: “alarm signals can be broadcast live on-site and/or sent to the handheld devices of maintenance or management personnel so that appropriate measures can be taken in a timely manner”).
Yang’s use of detection results from multiple images to determine when a dust alarm condition is satisfied provides a transition from a non-alarm condition to an alarm condition. Once the system determines that the current detection result causes the accumulated detection results to cross the alarm threshold, recording the acquisition time of that current image as the beginning of the detected dust event would provide a temporal indication of when the alarm condition commenced.
As per claim 5, claim 1 is incorporated and Yang in view of Ji teaches: wherein the detection data comprises a fugitive dust level indicated by the fugitive dust level information; and
said giving the fugitive dust warning in the case that the fugitive dust warning condition is satisfied based on the detection data and the historical detection data set comprises:
giving, in response to a preset warning mechanism being a real-time warning mechanism, the fugitive dust warning based on a fugitive dust level in the detection data and fugitive dust levels in the historical detection data set; and
giving, in response to the preset warning mechanism being an interval warning mechanism and a time difference between a current system time and a last warning time after the fugitive dust warning is given being greater than an interval warning duration, the fugitive dust warning based on the fugitive dust level in the detection data and the fugitive dust levels in the historical detection data set (see Yang, e.g., para. [0104]: “issuing an alarm when the number of target detection images is greater than an alarm threshold.”; para. [0078]: “when the dust concentration is greater than the concentration threshold, or the dust concentration level is greater than the concentration level threshold, the dust detection image can be identified as the target detection image. If the number of target detection images is greater than the alarm threshold, an alarm signal is generated…alarm signals can be broadcast live on-site and/or sent to the handheld devices…”; also see Ji, e.g., para. [0018]: “generating a third alarm message based on second location information of the construction dust and the category of the construction dust, to prompt processing of the construction dust based on the second location information in a manner matching the category of the construction dust. In this way, different alarm messages are generated based on the different detected contents, so as to provide timely feedback to the management personnel on the actual situation of the construction site.”).
As per claim 6, claim 2 is incorporated and Yang in view of Ji teaches: wherein after giving the fugitive dust warning, the method further comprises:
determining, in response to a sum of the state values in a preset number of historical detection data items being less than or equal to a second preset threshold, that fugitive dust in the preset collection region ends, and recording a fugitive dust end time (see Ji, e.g., para. [0010]-[0011], [0025]: discloses temporal-image detection).
Once the combined system uses accumulated historical detection results to determine onset of a dust condition, it would have been desirable to use the same temporal detection information to determine when the detected dust condition has subsided. Using a second threshold lower than or otherwise distinct from the alarm threshold provides a temporal criterion for determining termination of the detected event.
As per claim 7, claim 2 is incorporated and Yang in view of Ji teaches: wherein the first preset condition comprises that an area of the fugitive dust detection box is greater than or equal to a third preset threshold; and/or intersection over union of the fugitive dust detection box and a preset fugitive dust reference box is greater than or equal to a fourth preset threshold (see Yang, e.g., para. [0036]: “The target detection bounding box is used to mark the location of dust in the dust detection image, and the bounded area shows the dust area in the image”; para. [0048]-[0050]: “IOU is the intersection-union ratio, used to represent the overlap between the predicted detection box b and the ground truth detection box.”; Ji e.g., abstract; para. [0010]-[0011], [0072]-[0073], [0087]: discloses image features, detection of dust, semantic segmentation, location information, different dust/soil regions).
Ji teaches determining the location and extent of detected construction-site dust regions and using the resulting positional information in subsequent processing and warning generation. Applying a minimum region-size criterion and/or overlap criterion to the detected region would have been a predictable way of filtering detections based on their spatial correspondence to a reference region.
As per claim 8, claim 1 is incorporated and Yang in view of Ji teaches: wherein after giving the fugitive dust warning, the method further comprises:
returning to perform the step of sequentially collecting frame images from the video stream as detection images and acquiring the object detection result by inputting the detection images into the image recognition model, and in a case that the object detection result further comprises an exposed soil dreg detection box, determining whether the exposed soil dreg detection box satisfies a second preset condition based on positioning information of the exposed soil dreg detection box; and
giving, in response to each exposed soil dreg detection box satisfying the second preset condition within a first preset frame number range, an exposed soil dreg warning and generating exposed soil dreg warning information, wherein the exposed soil dreg warning information comprises a location of exposed soil dregs within the preset collection region (see Yang, e.g., para. [0080]: the device can also be used for dust suppression and removal at bulk cargo terminals to meet environmental protection requirements. It can also conduct area detection to better grasp the overall dust distribution; and Ji, e.g., abstract; para. [0010]-[0012], [0015], [0072]-[0073], [0086]-[0087], [0092]: teaches continuing detection using an image to be detected and temporally adjacent frames, detecting construction-site oil, determining location information, determining categories, and generating warning information based on the location/category).
Ji expressly teaches that, after detecting construction-site fugitive dust, construction-site soil is detected in the same image or in temporally adjacent images, and further teaches determining location information and generating warning information based on the detected soil. Thus, Ji provides a direct teaching for extending the dust-monitoring process of Yang to a second environmental condition represented by detected site soil and for using temporally successive images and positional information in generating the corresponding warning.
As per claim 9, claim 8 is incorporated and Yang in view of Ji teaches: wherein the second preset condition comprises that a number of the exposed soil dreg detection boxes is greater than or equal to a fifth preset threshold; and/or an area of the exposed soil dreg detection box is greater than or equal to a sixth preset threshold; and/or intersection over union of the exposed soil dreg detection box and a preset exposed soil dreg reference box is greater than or equal to a seventh preset threshold (see Ji, e.g., para. [0015], [0069], [0072]-[0075], [0085]-[0087]: teaches detecting site soil, location, classification, segmentation).
Ji’s temporal adjacent-frame processing and segmentation provide the underlying detection results, while applying minimum frame-count, region-area, and/or spatial overlap criteria for distinguishing persistent/relevant exposed soil from an isolated or insignificant detection.
As per claim 16, claim 2 is incorporated and Yang in view of Ji teaches: said the fugitive dust warning condition is satisfied based on the detection data and the historical detection data set comprises: acquiring a state value sum by accumulatively calculating a sum of state value in the detection data and state values in various historical detection data items in the historical detection data set; and determining that the fugitive dust warning condition is satisfied in a case that the state value sum is greater than or equal to a first preset threshold (see Ji, abstract; para. [0010]-[0011], [0072]-[0073], [0087]: discloses performing image-feature detection and segmentation and obtains location information; The particular selection of an area threshold and/or overlap threshold for determining whether a detected region constitutes dust would further refine the detection criterion based on the positional information already obtained from the detected region).
As per claim 17, claim 2 is incorporated and Yang in view of Ji teaches: wherein the historical detection data set is capable of containing no more than a preset number of historical detection data items; and said updating the historical detection data set based on the detection data comprises: removing, in a case that a number of the historical detection data items in the historical detection data set is equal to the preset number, a historical detection data item with earliest storage time from a current historical detection data set, and adding the detection data to the historical detection data set as a new historical detection data item (see Yang, e.g., para. [0079]: “the alarm threshold and/or concentration threshold of the image can be adjusted according to the detection index requirements…”; also see Ji, e.g., para. [0061]: “at least one frame of the image to be detected that is temporally adjacent to the image to be detected is acquired. For example, the previous frame of the image to be detected or the next frame of the image to be detected.”).
Because the system of Yang, as supplemented by Ji, processes a sequence of detection images and uses detection results from previously processed images, it would have been desirable to limit the amount of retained historical detection information to a predetermined number of entries. When the system reached its set limit, it could remove the oldest detection result before adding the newest one. This would create a standard sliding window that kept only the most recent results. It would preserve the information needed to determine alarms, prevent the stored history from growing endlessly, and make the system’s behavior predictable.
Claims 14 and 18-21 (directed to a computer device) correspond to method claims 1-5. Therefore, claims 14 and 18-21 are rejected for the same reasons of obviousness as corresponding method claims 1-5 above for having similar limitations and being similar in scope.
Claim 15 (directed to a non-transitory computer-readable storage medium) corresponds to method claim 1. Therefore, claim 15 is rejected for the same reasons of obviousness as corresponding method claim 1 above for having similar limitations and being similar in scope.
Claims 10 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Ji, and further in view of Ren et al. (Chinese Publication No. CN111652128A; machine translation obtained from Espacenet; hereinafter as “Ren”) and Wang (Chinese Publication No. CN114663809A; machine translation obtained from Espacenet).
As per claim 10, claim 1 is incorporated and Yang in view of Ji teaches: wherein after giving the fugitive dust warning, the method further comprises:
sending, in a case that a fugitive dust level indicated by the fugitive dust warning reaches a preset fugitive dust level, an instruction for persons to evacuate the preset collection region;
returning, in response to receiving an instruction to track the persons, to perform the step of sequentially collecting frame images from the video stream as detection images and acquiring the object detection result by inputting the detection images into the image recognition model, and in a case that the object detection result further comprises a person detection box, determining a number of persons in a preset evacuation reference box based on positioning information of the person detection box and positioning information of the preset evacuation reference box; and
giving, in a case that the number of persons is greater than or equal to an eighth preset threshold and an evacuation duration is greater than or equal to a preset evacuation duration, a person evacuation warning, wherein the evacuation duration is a difference between a current system time and an evacuation start time, the evacuation start time being a time responsive to receiving an instruction to track the persons (see Yang, e.g., para. [0078]: “alarm signals can be broadcast live on-site and/or sent to the handheld devices of maintenance or management personnel so that appropriate measures can be taken in a timely manner”; Ji e.g., abstract; para. [0025], [0046]-[0047], [0072]: the neural-network framework could detect different categories including dust, soil, site scene).
Yang in view of Ji does not expressly teach: personnel evacuation, person tracking, evacuation reference frame, evacuation duration, counting persons, evacuation alarm.
However, in the same field of endeavor, Ren discloses a high altitude power job safety monitoring method, and specifically discloses (e.g., para. [0021]-[0059]): feeding learning images into the convolutional neural network in advance to train the convolutional neural network to classify the learning images into at least three classes of human bodies, gloves, and utility poles, thereby making the convolutional neural network classify the fed image data; feeding the image data into the neural network for multi-class object detection to generate a human detection frame, a glove detection frame, and a utility pole detection frame on the image data. Wherein the steps of acquiring job site image data captured by a monitoring apparatus include acquiring video data captured by a monitoring apparatus provided at the job site, extracting a single-frame image of the video data, and generating the image data. As such, Ren discloses recognizing a plurality of different target objects by the same image recognition model and outputting detection boxes and classification classes. Based on this, it is conceivable for a person skilled in the art that dust detection and person detection are performed by the same image recognition model when detecting the environment of a worksite scene. Additionally, in the same field of endeavor, Wang discloses an employee status identification method and specifically discloses (e.g., paragraphs [0051]-[0107]): acquiring a surveillance video of a station to be identified; inputting the surveillance video to a pedestrian detection model to obtain pedestrian feature information; acquiring a target position to which the pedestrian feature information corresponds and a reference position to which the station to be identified corresponds, and determining a positional relationship between the target position and the reference position; if the target location is within the reference location, determining that the station to be identified is employee to employment status. Since the pedestrian feature information includes a bounding box for the pedestrian in the boxed-out image, thus, the target position of the pedestrian in the image can be determined from the bounding box, whereby, from the target position of the pedestrian in the image and the reference position of the station to be identified set in advance, it can be determined whether someone is at the station to be identified, thereby determining the status of employment to be reached by the station to be identified. At step 1061, the cropped video is input to the video behavior analysis model frame by frame to obtain employee status information for each frame of the cropped video. At step 107, upon reaching an alarm condition by the employee status analysis, an alarm message is sent according to a preset rule, comprising: alarm time as accumulated time since alarm emission, that is, alarm emission at two consecutive times is required to be separated by a preset time threshold, thereby effectively reducing the frequency of alarm emission and avoiding repeated alarms. It follows that, Wang discloses confirming whether a reference position is someone based on a reference box set in advance and a bounding box of pedestrian detection, based on this, in order to confirm whether or not and the number of people in a predetermined reference position in the jobsite environment, it is conceivable for a person skilled in the art to determine the number of people in the preset evacuation reference frame based on the positioning information of the person detection frame and the positioning information of the preset evacuation reference frame. Furthermore, when the number of personnel is greater than or equal to a certain number and the difference between the current time of the system and the start time of evacuation is greater than a certain time period, a personnel evacuation alarm is performed, as is conventional for those skilled in the art.
Accordingly, the subject matter of claim 10 would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
As per claim 11, claim 1 is incorporated and Yang in view of Ji teaches: wherein after giving the fugitive dust warning, the method further comprises:
returning to perform the step of sequentially collecting frame images from the video stream as detection images, and acquiring the object detection result by inputting the detection images into the image recognition model, and in a case that the object detection result further comprises a sign detection box, determining a matching result of the sign detection box and a preset sign reference box based on positioning information of the sign detection box and positioning information of the preset sign reference box; and
giving, in a case that the sign detection box does not match the preset sign reference box within a third preset frame number range, a sign warning and generating sign warning information, wherein the sign warning information comprises a location of a sign in the preset collection region (see Yang, e.g., para. [0057]: “obtaining a target detection frame output by the dust labeling model, wherein the target detection frame is used for marking a dust position”; Ji e.g., abstract; para. [0069]: The preset semantic tags in the preset semantic tag library are set according to the processing requirements carried in the image).
Yang in view of Ji does not expressly teach: personnel evacuation, person tracking, evacuation reference frame, evacuation duration, counting persons, evacuation alarm.
However, in the same field of endeavor, Ren discloses a high altitude power job safety monitoring method, and specifically discloses (para. [0021]-[0059]): feeding learning images into the convolutional neural network in advance to train the convolutional neural network to classify the learning images into at least three classes of human bodies, gloves, and utility poles, thereby making the convolutional neural network classify the fed image data; feeding the image data into the neural network for multi class object detection to generate a human detection frame, a glove detection frame, and a utility pole detection frame on the image data. Wherein the steps of acquiring job site image data captured by a monitoring apparatus include acquiring video data captured by a monitoring apparatus provided at the job site, extracting a single-frame image of the video data, and generating the image data. As such, Ren discloses identifying a plurality of different target objects by the same image recognition model and outputting detection boxes and classification classes. Based on this, it is conceivable for a person skilled in the art that dust detection and sign detection are performed by the same image recognition model when detecting the environment of a jobsite scene. Additionally, Wang discloses an employee status identification method and specifically discloses (para. [0051]-[0107]): acquiring a surveillance video of a station to be identified; inputting the surveillance video to a pedestrian detection model to obtain pedestrian feature information; acquiring a target position to which the pedestrian feature information corresponds and a reference position to which the station to be identified corresponds, and determining a positional relationship between the target position and the reference position; if the target location is within the reference location, determining that the station to be identified is employee to employment status. Since the pedestrian feature information includes a bounding box for the pedestrian in the boxed-out image, thus, the target position of the pedestrian in the image can be determined from the bounding box, whereby, from the target position of the pedestrian in the image and the reference position of the station to be identified set in advance, it can be determined whether someone is at the station to be identified, thereby determining the status of employment to be reached by the station to be identified. At step 1061, the cropped video is input to the video behavior analysis model frame by frame to obtain employee status information for each frame of the cropped video. At step 107, upon reaching an alarm condition by the employee status analysis, an alarm message is sent according to a preset rule, comprising: alarm time as accumulated time since alarm emission, that is, alarm emission at two consecutive times is required to be separated by a preset time threshold, thereby effectively reducing the frequency of alarm emission and avoiding repeated alarms. It follows that, Wang discloses confirming whether a target object is present in a reference position based on a reference box set in advance and a bounding box of a target object detection, based on this, determining a matching result of the sign detection box with the preset sign reference box, based on the location information of the sign detection box and the location information of the preset sign reference box, confirms the sign status, as would occur to one of skill in the art. In addition, when a condition is satisfied based on the consecutive multi-frame detection result, alarming has been disclosed in Yang, based on which if none of the sign detection frames matches the preset sign reference frames within a third preset frame number range, then sign alarm is performed and sign alarm information is generated while giving the position of the sign, as will be apparent to those skilled in the art.
Accordingly, the subject matter of claim 11 would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim 12 is rejected under 35 U.S.C. 103 as being unpatentable over Yang in view of Ji, and further in view of Ren et al. (Chinese Publication No. CN111652128A; machine translation obtained from Espacenet; hereinafter as “Ren”).
As per claim 12, claim 1 is incorporated and Yang in view of Ji teaches: wherein before inputting the detection images into an image recognition model, the method further comprises:
training the image recognition model,
said training the images recognition model comprises:
acquiring multi-frame sample images of the preset collection region, and labeling the sample images with sample labels, wherein the sample label comprises location information of at least one reference box corresponding to the preset collection region and category information of each reference box, the category information comprising one of a weather category, a person category, a sign category, and an exposed soil dreg category;
training the image recognition model to be trained based on the sample images and the sample labels; and
constructing a weighted loss value and continuously training the image recognition model by performing weighted backpropagation on the weighted loss value until the weighted loss value converges to acquire a trained image recognition model (see Yang, e.g., para. [0007]-[0008]: “training the YOLOv3-SPP model using the sample data to obtain the dust annotation model” and “using the CIoU loss function as the regression loss function, training the YOLOv3-SPP model to obtain a dust labeling model”; and Ji, e.g., para. [0069], [0077], [0133]-[0134]).
Yang in view of Ji does not expressly teach: personnel evacuation, person tracking, evacuation reference frame, evacuation duration, counting persons, evacuation alarm.
However, in the same field of endeavor, Ren discloses a high altitude power job safety monitoring method, and specifically discloses (e.g., para. [0021]-[0059]): feeding learning images into the convolutional neural network in advance to train the convolutional neural network to classify the learning images into at least three classes of human bodies, gloves, and utility poles, thereby making the convolutional neural network classify the fed image data; feeding the image data into the neural network for multi-class object detection to generate a human detection frame, a glove detection frame, and a utility pole detection frame on the image data. Wherein the steps of acquiring job site image data captured by a monitoring apparatus include acquiring video data captured by a monitoring apparatus provided at the job site, extracting a single-frame image of the video data, and generating the image data. As such, Ren discloses training a neural network model to achieve class and border detection of different classes of objects by the same image recognition model based on multiple classes of training samples and class and detection box labels. Based thereon, for detection of one of a weather category, a people category, a sign category, and an exposed soil dreg category in the environment, multiple frame sample images of a preset acquisition area are acquired and sample labels are annotated for the sample images; the sample label comprises position information of at least one reference box corresponding to the pre-set acquisition area, and class information of each of the reference boxes, and training an image recognition model to be trained on the basis of the sample images and the sample labels is conceivable to a person skilled in the art. In addition, multi-objective image classification regression is used to construct weighted loss values and perform weighted back propagation to continue training the image recognition model until the weighted loss values converge.
Accordingly, the subject matter of claim 12 would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of References Cited for a listing of analogous art.
Karnik et al. (U.S. Publication No. 20190346356A1) discloses systems and methods for monitoring air particulate matter.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADNAN AZIZ whose telephone number is (571) 270-7536, (FAX: 571-270-8536). The examiner can normally be reached Monday - Friday (9am - 6pm Eastern Time).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, QUAN-ZHEN WANG can be reached at 571-272-3114. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADNAN AZIZ/Primary Examiner, Art Unit 2685 adnan.aziz@uspto.gov
Prosecution Insights