The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Status of the Claims
This is in response to applicant’s amendment filed 6/1/26. Claims 1-20 are pending in the application.
Continuing Examination
A request for continued examination under 37 CFR 1.114, including the fee set forth in37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 7/6/26 has been entered.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim Rejections - 35 USC § 102
Claims 1, 8 and 15-16 are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Wang et al. (Wang; CN 109977854 A).
Regarding Claim 1, Wang discloses a method for detecting an entrapment situation inside an elevator car (Abstract), the method comprising the steps of:
receiving an inoperable notification indicating an inoperable condition of the elevator car (Summary video analysis classification unit analyzes and classifies the pre-processed data… abnormal behavior of the passengers who occur in the elevator, such as…infant is trapped in the elevator, Claim 1);
obtaining from at least one imaging device arranged inside the elevator car real-time image data of an interior of the elevator car in response to receiving the inoperable notification (Summary video analysis classification unit analyzes and classifies the pre-processed data, and obtains whether an abnormal behavior occurs in the elevator at the current time in real time; Claim 1);
detecting one or more human objects inside the elevator car by performing a detection procedure based on the obtained real-time image data (Summary abnormal behavior of the passengers who occur in the elevator, such as the old man fainting in the elevator, the infant is trapped in the elevator, the woman is harassed) and at least one previously generated reference image (Summary step 102 Video training step: When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected for modeling training; modeling training obtains the anomaly coefficient of each video, thereby obtaining the example score and negative of the positive example video segment. The example score of the example video segment, and then the corresponding loss value according to the loss function, and the network learning model is established according to the size of the loss value; The feature of the video clip is extracted by the model) ,
wherein generating of the at least one reference image comprises:
obtaining from the at least one imaging device random image data of the interior of the elevator car (Detailed Description 102 Video training step: During the data training process, 30 positive example packages and 30 negative example packages are randomly selected for modeling training, Claim 2), wherein the random image data comprises data from a plurality of images (Summary step 101 Training preparation steps…obtain the abnormal video stream and the normal video stream segmentation, wherein each video partition is a video segment of 5 seconds/segment, and each segment of the video is evenly divided into 32 segments as a package; Claim 2) captured in a plurality of random reference scenarios (Summary step 102 Video training step: When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected for modeling training); and
processing the obtained random image data to generate the at least one reference image (Summary step 102 Video training step: When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected for modeling training; modeling training obtains the anomaly coefficient of each video, thereby obtaining the example score and negative of the positive example video segment. The example score of the example video segment, and then the corresponding loss value according to the loss function, and the network learning model is established according to the size of the loss value; The feature of the video clip is extracted by the model; Claim 2 ); and
generating, to an elevator control system and/or to a service center, a signal indicating a detection of the entrapment situation in response to the detecting the one or more human objects (Claim 1 alarm playing unit determines, according to the result of receiving the video analysis classification unit, a warning video or a comfort video that needs to be matched in the abnormal behavior in the elevator).
Regarding the new limitation of wherein the random image data comprises data from a plurality of images captured in a plurality of random reference scenarios occurring at different times of day or different days, Applicant has argued that, “As disclosed on page 11 of the Specification, "The random image data comprises a plurality of images captured in a plurality of random reference scenarios. In other words, there is no specific or predefined point of time, time of day, date, or situation, when each of the plurality of the images of the random image data is captured, and there is no specific or predefined time interval between capturing the images of the plurality of images of the random image data."
Using the definition provided by Applicant’s Specification, Wang also teaches random reference scenarios occurring at different times of day or different days (Summary step 102 30 positive example packages and 30 negative example packages are randomly selected for modeling training; Claim 2). Wang teaches each video partition is a video segment of 5 seconds/segment, and each segment of the video is evenly divided into 32 segments as a package. After the normal video stream is segmented as a positive example video segment, the abnormal video stream is segmented as a negative example video segment. When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected. Because each package occurs at different times, the random scenario packages selected occur at different times of day.
Regarding Claim 8, Wang discloses an elevator computing unit for detecting an entrapment situation inside an elevator car (Abstract), the elevator computing unit comprising:
a processing unit comprising at least one processor ; and a memory unit comprising at least one memory including computer program code, wherein the at least one memory and the computer program code are configured to, with the at least one processor (Summary An abnormal behavior detection and analysis system in an elevator monitoring environment, comprising a camera, a monitoring video acquisition unit, a video analysis classification unit, a video result processing unit, an alarm playing unit, and an in-air elevator video playing unit), cause the elevator computing unit to:
receive an inoperable notification indicating an inoperable condition of the elevator car (Summary video analysis classification unit analyzes and classifies the pre-processed data; Claim 1);
obtain from at least one imaging device arranged inside the elevator car real-time image data of the interior of the elevator car in response to receiving the inoperable notification (Summary video analysis classification unit analyzes and classifies the pre-processed data, and obtains whether an abnormal behavior occurs in the elevator at the current time in real time; Claim 1); and
detect one or more human objects inside the elevator car by performing a detection procedure based on the obtained real-time image data (Summary abnormal behavior of the passengers who occur in the elevator, such as the old man fainting in the elevator, the infant is trapped in the elevator, the woman is harassed) and at least one previously generated reference image (Summary step 102 Video training step: When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected for modeling training; modeling training obtains the anomaly coefficient of each video, thereby obtaining the example score and negative of the positive example video segment. The example score of the example video segment, and then the corresponding loss value according to the loss function, and the network learning model is established according to the size of the loss value; The feature of the video clip is extracted by the model),
wherein to generate the at least one reference image the elevator computing unit is configured to:
obtain from the at least one imaging device random image data of the interior of the elevator car (Detailed Description 102 Video training step: During the data training process, 30 positive example packages and 30 negative example packages are randomly selected for modeling training, Claim 2), wherein the random image data comprises data from a plurality of images (Summary step 101 Training preparation steps…obtain the abnormal video stream and the normal video stream segmentation, wherein each video partition is a video segment of 5 seconds/segment, and each segment of the video is evenly divided into 32 segments as a package; Claim 2) captured in a plurality of random reference scenarios (Summary step 102 Video training step: When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected for modeling training); and
process the obtained random image data to generate the at least one reference image (Summary step 102 Video training step: When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected for modeling training; modeling training obtains the anomaly coefficient of each video, thereby obtaining the example score and negative of the positive example video segment. The example score of the example video segment, and then the corresponding loss value according to the loss function, and the network learning model is established according to the size of the loss value; The feature of the video clip is extracted by the model; Claim 2); and
generate, to an elevator control system and/or to a service center, a signal indicating a detection of the entrapment situation in response to the detecting the one or more human objects (Claim 1 alarm playing unit determines, according to the result of receiving the video analysis classification unit, a warning video or a comfort video that needs to be matched in the abnormal behavior in the elevator).
Regarding the new limitation, wherein the random image data comprises data from a plurality of images captured in a plurality of random reference scenarios occurring at different times of day or different days, Applicant has argued that, “As disclosed on page 11 of the Specification, "The random image data comprises a plurality of images captured in a plurality of random reference scenarios. In other words, there is no specific or predefined point of time, time of day, date, or situation, when each of the plurality of the images of the random image data is captured, and there is no specific or predefined time interval between capturing the images of the plurality of images of the random image data."
Using the definition provided by Applicant’s Specification, Wang also teaches random reference scenarios occurring at different times of day or different days (Summary step 102 30 positive example packages and 30 negative example packages are randomly selected for modeling training; Claim 2). Wang teaches each video partition is a video segment of 5 seconds/segment, and each segment of the video is evenly divided into 32 segments as a package. After the normal video stream is segmented as a positive example video segment, the abnormal video stream is segmented as a negative example video segment. When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected. Because each package occurs at different times, the random scenario packages selected occur at different times of day.
Regarding Claim 15, Wang discloses a detection system for detecting an entrapment situation inside an elevator car (Abstract), the detection system comprising: at least one imaging device arranged inside the elevator car (Summary An abnormal behavior detection and analysis system in an elevator monitoring environment, comprising a camera, a monitoring video acquisition unit, a video analysis classification unit, a video result processing unit, an alarm playing unit, and an in-air elevator video playing unit); and the elevator computing unit according to claim 8.
Regarding Claim 16, Wang discloses computer executable instructions which, whenAn abnormal behavior detection and analysis system in an elevator monitoring environment, comprising a camera, a monitoring video acquisition unit, a video analysis classification unit, a video result processing unit, an alarm playing unit, and an in-air elevator video playing unit), cause the .
Claim Rejections - 35 USC § 103
Claims 2, 6, 9, 13 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Wang.
Regarding Claims 2 and 9, Wang discloses the detection procedure comprises: an object detection phase comprising detecting one or more abnormal behavior of the passengers who occur in the elevator, such as the old man fainting in the elevator, the infant is trapped in the elevator, the woman is harassed), but doesn’t specify a separate object detection.
However, as the system obtains can discern different scenarios of humans in distress, it has to discern between inanimate objects and humans, to train a model, it would eventually obtain a video of objects not human.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Wang with using scenarios with objects in order to create a robust sample set improving accurate detection of presence in the elevator.
Regarding Claims 6, 13 and 18, Wang discloses the plurality of reference scenarios comprises at least multiple Video training step: When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected for modeling training; modeling training obtains the anomaly coefficient of each video, thereby obtaining the example score and negative of the positive example video segment. The example score of the example video segment, and then the corresponding loss value according to the loss function, and the network learning model is established according to the size of the loss value; The feature of the video clip is extracted by the model). But doesn’t specify empty elevator scenarios. However, as the system obtains randomly selected video clips, to train a model, it would eventually obtain a video of an empty elevator.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Wang with using scenarios with empty elevators in order to create a robust sample set improving accurate detection of presence in the elevator.
Claims 7 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Wang in view of Lienard et al. (Lienard; US 20070058780 A1).
Regarding Claims 7 and 14, Wang discloses the processing of the obtained random image data, but doesn’t teach performing a median operation on pixel values of the plurality of images of the random image data to generate the at least one reference image.
Lienard teaches performing a median operation on pixel values of the plurality of images of the random image data to generate the at least one reference image ([0034] processing comprises a spatial filtering operation of the pixels of class 1, for example by using a non-linear filter, for example a median filter or a bilateral filter. Such filters are particularly adapted to preserve the contours).
Lienard applies a known technique (median operations) to a known device (imaging systems) ready for improvement to yield predictable results, by adding a median operation applicable from one system onto another conventional system for image processing.
The adaptation would yield predictable results, providing improved processing of images to reduce noise, as suggested by Lienard ([0006]).
Allowable Subject Matter
Claim 3-5, 10-12, 17 and 19-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Response to Arguments
Applicant's arguments filed 6/1/26 have been fully considered but they are not persuasive for the following reasons:
A. Applicant respectfully submits that this combination of elements as set forth in independent claim 1 is not disclosed or made obvious by the prior art of record, including Wang.
Wang discloses a system for determining an abnormal situation in an elevator car such as people trapped in the elevator. The system includes obtaining a reference image by obtaining 30 abnormal video streams and 30 normal video streams. Each of these streams is provided a score and only the example with the highest score is used. The video stream with the highest score is considered to be a plurality of random images. The video of Wang is for training purposes and is not a reference image. Wang does not refer to generating a reference image but rather calculating a loss value and refers to convolution layers, pooling layers and convolution kernels. Moreover, even assuming the modeling training generates a reference image, the modeling training uses a single video segment, not data from images captured in random reference scenarios.
It is respectfully submitted that a video segment (as taught by Wang Claim 2) is made of a plurality of images, and therefore reads on the claimed reference image. The video segment selected for training, reads on the reference image. Although a video is disclosed, a video is made up of a number of images.
B. Applicant argues that although the video of Wang can be considered a plurality of images, there is no disclosure that images captured in a plurality of random reference scenarios is used to obtain the score to determine which video stream will be used.
It is respectfully submitted that, Wang teaches random reference scenarios (Summary step 102 Video training step: When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected for modeling training).
C. Applicant argues that claim 1 now recites a plurality of random reference scenarios occurring at different times of day or different days. As disclosed on page 11 of the Specification, "The random image data comprises a plurality of images captured in a plurality of random reference scenarios. In other words, there is no specific or predefined point of time, time of day, date, or situation, when each of the plurality of the images of the random image data is captured, and there is no specific or predefined time interval between capturing the images of the plurality of images of the random image data." The use of images without respect to specific or predefined point of time, time of day, date, or situation distinguishes the claimed plurality of images from the video of Wang.
It is respectfully submitted that, using the definition provided by Applicant, Wang also teaches random reference scenarios occurring at different times of day or different days (Summary step 102 30 positive example packages and 30 negative example packages are randomly selected for modeling training; Claim 2). Wang teaches each video partition is a video segment of 5 seconds/segment, and each segment of the video is evenly divided into 32 segments as a package. After the normal video stream is segmented as a positive example video segment, the abnormal video stream is segmented as a negative example video segment. When data training is performed, 30 positive example packages and 30 negative example packages are randomly selected. Because each package occurs at different times, the random scenario packages selected occur at different times of day.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
a. Tadashi (JP 2003040541 A) discloses a shutting-in monitoring device for an elevator capable of detecting shutting-in with high reliability using a monitoring camera. An image of a floor surface of a cage in the elevator is picked up by the monitoring camera. An average value of brightness in a partial area 21 in the image is computed, and a set of image data is selected from plural sets of reference image data based on the average value. The selected set of the reference image data is compared with inputted image data, so that if an object exists in the cage or not is determined. Four kinds of the reference image data, for bright time, moderately bright time, dark time, and direct sunbeam time, are stored. Tadashi discloses random image data comprises data from a plurality of images captured in a plurality of random reference scenarios occurring at different times of day or different days (Abstract, page 4).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MARK S RUSHING whose telephone number is (571)270-5876. The examiner can normally be reached on 10-6pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Davetta Goins can be reached at 571-272-2957. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MARK S RUSHING/Primary Examiner, Art Unit 2689
Prosecution Insights