DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-16 are pending.
Priority
The present application claims priority to, and the benefit of, U.S. Provisional Patent Application No. 63/125,975 filed on December 15, 2020.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 7-8, and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of ANBAI (US 20090240984 A1 Hereinafter “ANBAI”) in further view of Rohanna (US 4438831 A Hereinafter “Rohanna”).
Rejection overview: Lin teaches the idea of an elevator monitoring system which captures image of individuals in elevators to determine if the elevator is at capacity, and if the elevator is at capacity, putting the elevator into an operating mode where it ignores floors calls and bypasses floors to let people off for efficient elevator loading and unloading. Lin is largely silent about the typical hardware associated with an elevator, and also silent about the structure of the computer implementation; a rational conclusion from this silence is that these gaps are filled with known solutions in the art. Strong teaches that an ASIC programmable to implement a DNN can be used to detect people in captured images like the one used for determining the elevator capacity. Rohanna teaches the concept that the activation of bypass mode in an elevator can come from activation of a relay from a low voltage state to a high voltage state, and ANBAI teaches that a general purpose input output (GPIO) can be used to perform the driving from low voltage to high voltage (activation of the relay).
Regarding claim 1, Lin teaches a sensor module comprising:
a camera module comprising an image processor and a lens that collectively capture image data representative of a field of view (FOV) a scene (Fig. 1B, [0022]: “As shown in FIG. 1B, additional video camera 32 located in elevator cab 18 provides video input with respect to the interior of elevator cab 18 to video processor 16”. The video camera has a lens and processor to take video, and these work together to capture images of a FOV of a scene (cab). The video captures a sequence of frames similar to images and the processor processes there frames similar to how an image processor processes images);
;
at least one processor (Fig. 1B, [0019]: “Video processor 16 provides passenger data to control system 24, providing control system 24 with additional information regarding elevator passengers”); and
a non-transitory storage medium storing instructions thereon that, upon execution by the at least one processor (Fig. 1B, [0019]: “Video processor 16 provides passenger data to control system 24, providing control system 24 with additional information regarding elevator passengers”. The instructions for processing the video on the processor would be stored on a non-transitory storage medium storing instructions), performs operations comprising:
capturing, by the camera module, image data of the FOV of the camera module (Fig. 1B, [0022]: “As shown in FIG. 1B, additional video camera 32 located in elevator cab 18 provides video input with respect to the interior of elevator cab 18 to video processor 16”);
detecting, based on the captured image data, the presence of one or more persons within the FOV of the camera (Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected);
counting, by the at least one processor, a number of persons detected within the FOV of the camera (Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected); and
based on the counted number of persons exceeding a threshold(Fig. 1B, [0022]: “For example, if video processor 16 determines that elevator cab 18 contains no available space for additional passengers, then control system 24 causes elevator cab 18 to bypass floors with waiting passengers. This prevents the situation in which an elevator filled to capacity stops at a floor, increasing the ride time of passengers within the elevator cab, and the wait time for passengers waiting for an elevator, since they must now wait for another elevator to be dispatched to their floor”. The threshold for the counted number of persons is the maximum capacity, and if the maximum capacity is exceeded, there is no available space.
Lin does not expressly disclose driving from low to high voltage based on the detected people to put the elevator in a bypass mode.
However, Rohanna teaches driving from low to high voltage based on the detected people to put the elevator in a bypass mode (Col. 6, lines 30-35: “Relay BP includes contacts BP-2 in the hall call circuits 108, with the change in their condition when relay BP is energized, preventing the elevator car from answering hall calls”)
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Lin’s elevator system to include Rohanna’s hardware for activating bypass mode alongside the driving from low to high voltage because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Rohanna’s hardware for activating bypass mode alongside the driving from low to high voltage permits putting the elevator in bypass mode using defined hardware and activation requirements. This known benefit in Rohanna is applicable to Lin’s elevator system as they both share characteristics and capabilities, namely, they are directed to activation of a bypass mode in elevators. Therefore, it would have been recognized that modifying in’s elevator system to include Rohanna’s hardware for activating bypass mode alongside the driving from low to high voltage would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Rohanna’s hardware for activating bypass mode alongside the driving from low to high voltage in activation of a bypass mode in elevators and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Lin does not expressly disclose using an ASIC programmable to implement a DNN for detecting the people in the cab.
However, Strong teaches an ASIC programmable to implement a DNN for detecting the people ([00376]: “In the illustrated example, pixel-domain CNN 3400 performs object detection and classification for visual analytics using data in the pixel or image domain (e.g., using decompressed visual data)”. These object are said are said to be alternative with people “vision operations (e.g., person or object detection, facial recognition” [0160]. The implementations of these processes on processors are also said to be on ASICs “In various embodiments, object recognition processor 6613, semantic processor 6614, and inference processor 6615 may be implemented using any suitable combination of hardware and/or software logic, and may further be implemented as separate logical or physical components or combined into one or more integrated components. In some embodiments, for example, processors 6613-6615 may be implemented using general-purpose central processing units (CPUs), graphics processing units (GPUs), and/or special-purpose processors designed for artificial intelligence, machine learning, and/or neural network applications (e.g., using application-specific integrated circuits (ASICs) or field-programmable gate array (FPGAs)), among other examples” [0679]).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Lin’s passenger counting system to include Strong’s CNN on an ASIC for object detection because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Strong’s CNN on an ASIC for object detection permits high object detection accuracy ([0279]: These topologies require millions of images to converge with high accuracy). This known benefit in Strong is applicable to Lin’s passenger counting system as they both share characteristics and capabilities, namely, they are directed to detecting objects in images (frames are similar to images). Therefore, it would have been recognized that modifying Lin’s passenger counting system to include Strong’s CNN on an ASIC for object detection would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Strong’s CNN on an ASIC for object detection in detecting objects in images (frames are similar to images) and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Lin does not expressly disclose a general purpose input output (GPIO) for controlling the elevator.
However, ANBAI teaches a general purpose input output (GPIO) selectively controllable to output at least one of a low voltage state and a high voltage state ([0039]: “Also, the GPIO 220a closes the relay 210a when a value of "1" is written in channel ch1 by the I2C control unit 130a. On the other hand, the GPIO 220a opens the relay 210a when a value of "0" is written in channel ch1”).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Lin’s control system to include ANBAI’s GPIO capable of activating a relay because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, ANBAI’s GPIO capable of activating a relay permits a device for driving low voltage to high voltage (activating a relay). This known benefit in ANBAI is applicable to Lin’s control system as they both share characteristics and capabilities, namely, they are directed to activation of relays for electronic processes. Therefore, it would have been recognized that modifying Lin’s control system to include ANBAI’s GPIO capable of activating a relay would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate ANBAI’s GPIO capable of activating a relay in activation of relays for electronic processes and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Regarding claim 2, the combination of Lin, Strong, ANBAI, and Rohanna teaches the sensor module of claim 1, in addition, Rohanna further teaches further comprising: a relay (Col. 6, lines 30-35: “Relay BP includes contacts BP-2 in the hall call circuits 108, with the change in their condition when relay BP is energized, preventing the elevator car from answering hall calls”)
The rationale for this combination is similar to the rationale for the Rohanna combination in the claim 1 rejection due to similar methods of combination and benefits.
ANBAI further teaches a general purpose input output (GPIO) selectively controllable to output at least one of a low voltage state and a high voltage state ([0039]: “Also, the GPIO 220a closes the relay 210a when a value of "1" is written in channel ch1 by the I2C control unit 130a. On the other hand, the GPIO 220a opens the relay 210a when a value of "0" is written in channel ch1”).
The rationale for this combination is similar to the rationale for the ANBAI combination in the claim 1 rejection due to similar methods of combination and benefits.
Regarding claim 7, the combination of Lin, Strong, ANBAI, and Rohanna teaches the sensor module of claim 1, in addition, Lin further teaches further comprising: (Fig. 1A: The communication between the video processor and the control system), wherein the non-transitory storage medium stores further instructions thereon that, upon execution by the at least one processor, performs additional operations comprising:
receiving,
the sensor module, wherein (Fig. 1B, [0022]: “For example, if video processor 16 determines that elevator cab 18 contains no available space for additional passengers, then control system 24 causes elevator cab 18 to bypass floors with waiting passengers. This prevents the situation in which an elevator filled to capacity stops at a floor, increasing the ride time of passengers within the elevator cab, and the wait time for passengers waiting for an elevator, since they must now wait for another elevator to be dispatched to their floor”. The threshold for the counted number of persons is the maximum capacity, and if the maximum capacity is exceeded, there is no available space. This leads to activation of bypass mode).
Rohanna further teaches the driving from low to high voltage (Col. 6, lines 30-35: “Relay BP includes contacts BP-2 in the hall call circuits 108, with the change in their condition when relay BP is energized, preventing the elevator car from answering hall calls”)
The rationale for this combination is similar to the rationale for the Rohanna combination in the claim 1 rejection due to similar methods of combination and benefits.
ANBAI further teaches a general purpose input output (GPIO) selectively controllable to output at least one of a low voltage state and a high voltage state ([0039]: “Also, the GPIO 220a closes the relay 210a when a value of "1" is written in channel ch1 by the I2C control unit 130a. On the other hand, the GPIO 220a opens the relay 210a when a value of "0" is written in channel ch1”).
The rationale for this combination is similar to the rationale for the ANBAI combination in the claim 1 rejection due to similar methods of combination and benefits.
Strong additionally teaches use of a transceiver ([0122]: “The mesh transceiver 562 may communicate using multiple standards or radios for communications at different range”).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Lin, Strong, ANBAI, and Rohanna’s communication hardware to include Strong’s use of a transceiver because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Strong’s use of a transceiver permits wireless communication between components in a system. This known benefit in Strong is applicable to the combination of Lin, Strong, ANBAI, and Rohanna’s communication hardware as they both share characteristics and capabilities, namely, they are directed to sending video data to other components for use. Therefore, it would have been recognized that modifying the combination of Lin, Strong, ANBAI, and Rohanna’s communication hardware to include Strong’s use of a transceiver would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Strong’s use of a transceiver in sending video data to other components for use and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Regarding claim 8, the combination of Lin, Strong, ANBAI, and Rohanna teaches the sensor module of claim 1, in addition, Strong further teaches wherein the DNN is a convolutional neural network that performs object detection ([00376]: “In the illustrated example, pixel-domain CNN 3400 performs object detection and classification for visual analytics using data in the pixel or image domain (e.g., using decompressed visual data)”).
The rationale for this combination is similar to the rationale for the Strong combination in the claim 1 rejection due to similar methods of combination and benefits.
Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of ANBAI (US 20090240984 A1 Hereinafter “ANBAI”) in further view of Rohanna (US 4438831 A Hereinafter “Rohanna”) in further view of Chen (US 20190304102 A1 Hereinafter “Chen”).
Regarding claim 3, the combination of Lin, Strong, ANBAI, and Rohanna teaches the sensor module of claim 1, in addition Lin further teaches wherein the non-transitory storage medium stores further instructions thereon that, upon execution by the at least one processor, performs additional operations comprising: determining, based on the detected presence of the one or more persons, (Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected).
The combination of Lin, Strong, ANBAI, and Rohanna does not expressly disclose applying trackers to the detected objects and counting the tracker to count the objects.
However, Chen teaches applying trackers to the detected objects and counting the tracker to count the objects ([0067]: “As described in more detail herein, a video analytics system can generate and detect foreground blobs that can be used to perform various operations, such as object tracking (also called blob tracking) and/or the other operations described above. A blob tracker (also referred to as an object tracker) can be used to track one or more blobs in a video sequence using one or more bounding boxes”. It is said that these video analytics like blob tracking can be used for people counting “Video analytics can also act as an intrusion detector, a video counter (e.g., by counting people, objects, vehicles, or the like), a camera tamper detector, an object left detector, an object/asset removal detector, an asset protector, a loitering detector, and/or as a slip and fall detector” ([0066])).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Lin, Strong, ANBAI, and Rohanna’s person detection system to include Chen’s person trackers because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Chen’s person trackers permits accurate detecting, tracking, and counting of detected individuals in the FOV of a video camera. This known benefit in Chen is applicable to the combination of Lin, Strong, ANBAI, and Rohanna’s person detection system as they both share characteristics and capabilities, namely, they are directed to counting people in video. Therefore, it would have been recognized that modifying the combination of Lin, Strong, ANBAI, and Rohanna’s person detection system to include Chen’s person trackers would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Chen’s person trackers in counting people in video and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of ANBAI (US 20090240984 A1 Hereinafter “ANBAI”) in further view of Rohanna (US 4438831 A Hereinafter “Rohanna”) in further view of Chen (US 20190304102 A1 Hereinafter “Chen”) in further view of Huang (US 20220188554 A1 Hereinafter “Huang”).
Regarding claim 4, the combination of Lin, Strong, ANBAI, Rohanna, and Chen teaches the sensor module of claim 3, in addition, Strong further teaches wherein the ML ASIC is a first ML ASIC, wherein the DNN is a first DNN, and wherein the sensor module further comprises:
in addition, Lin further teaches determining, based on the captured image data and the detected presence of one or more persons within the FOV of the camera,(Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected); and
determining the one or more persons based on the detected presence of the one or more persons(Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected).
Strong further teaches an ASIC programmable to implement a DNN for detecting the people ([00376]: “In the illustrated example, pixel-domain CNN 3400 performs object detection and classification for visual analytics using data in the pixel or image domain (e.g., using decompressed visual data)”. These object are said are said to be alternative with people “vision operations (e.g., person or object detection, facial recognition” [0160]. The implementations of these processes on processors are also said to be on ASICs “In various embodiments, object recognition processor 6613, semantic processor 6614, and inference processor 6615 may be implemented using any suitable combination of hardware and/or software logic, and may further be implemented as separate logical or physical components or combined into one or more integrated components. In some embodiments, for example, processors 6613-6615 may be implemented using general-purpose central processing units (CPUs), graphics processing units (GPUs), and/or special-purpose processors designed for artificial intelligence, machine learning, and/or neural network applications (e.g., using application-specific integrated circuits (ASICs) or field-programmable gate array (FPGAs)), among other examples” [0679]).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Lin’s passenger counting system to include Strong’s CNN on an ASIC for object detection because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Strong’s CNN on an ASIC for object detection permits high object detection accuracy ([0279]: These topologies require millions of images to converge with high accuracy). This known benefit in Strong is applicable to Lin’s passenger counting system as they both share characteristics and capabilities, namely, they are directed to detecting objects in images (frames are similar to images). Therefore, it would have been recognized that modifying Lin’s passenger counting system to include Strong’s CNN on an ASIC for object detection would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Strong’s CNN on an ASIC for object detection in detecting objects in images (frames are similar to images) and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
The combination of Lin, Strong, ANBAI, Rohanna, and Chen does not expressly disclose use of a second DNN and use of feature vectors for object detection
However, Huang teaches use of a second DNN and use of feature vectors for object detection (Claim. 2: “The method of claim 1, wherein generating the estimated 2D position of the object further comprises: determining, by a first neural network of the 2D-detection model, a first set of feature vectors from the image of the scene, the first set of feature vectors corresponding to the object in the image of the scene; and generating, by a 2D detector of the 2D-detection model, the estimated 2D position of the object in the image of the scene based on the first set of feature vectors received from the first neural network and a second set of feature vectors received from a second neural network of the 3D-detection model, the second set of feature vectors corresponding to the object in the depth measurements of the scene” (Emphasis added). Given Strong already disclosed use of a CNN on an ASIC for object detection, it would have been obvious to use it for the second neural network describes in Huang).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Lin, Strong, ANBAI, Rohanna, and Chen’s person detection system to include Huang’s use of a second neural network and feature vectors because such a modification is taught, suggested, or motivated by the art. More specifically, the motivation to modify the combination of Lin, Strong, ANBAI, Rohanna, and Chen to include Huang is expressly provided by Huang, stating that their invention provides “learning across 2D and 3D pipelines for improved object detection” ([0001]). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify the combination of Lin, Strong, ANBAI, Rohanna, and Chen’s person detection system to include Huang’s use of a second neural network and feature vectors with the motivation of improving object detection. The person of ordinary skill in the art would have recognized the benefit of improved object detection.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of ANBAI (US 20090240984 A1 Hereinafter “ANBAI”) in further view of Rohanna (US 4438831 A Hereinafter “Rohanna”) in further view of Dinh (US 20210272308 A1 Hereinafter “Dinh”).
Regarding claim 9, the combination of Lin, Strong, ANBAI, and Rohanna teaches the sensor module of claim 1, wherein the DNN is a convolutional neural network
The combination of Lin, Strong, ANBAI, and Rohanna does not expressly disclose a convolutional neural network that perform image segmentation.
However, Dinh teaches a convolutional neural network that perform image segmentation in automated capacity management ([0035]: “Accordingly, at least one embodiment includes automated capacity management using AI techniques. Such an embodiment includes implementing an intelligent IoT-based capacity management system using 3D depth sensor cameras and semantic image segmentation (e.g., instance-based segmentation) with AI and/or machine learning techniques).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Lin, Strong, ANBAI, and Rohanna’s convolutional neural network to include Dinh’s convolutional neural network that performs object segmentation because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Dinh’s convolutional neural network that performs object segmentation permits identification of object through segmentation for capacity management. This known benefit in Dinh is applicable to the combination of Lin, Strong, ANBAI, and Rohanna’s convolutional neural network as they both share characteristics and capabilities, namely, they are directed to capacity management. Therefore, it would have been recognized that modifying the combination of Lin, Strong, ANBAI, and Rohanna’s convolutional neural network to include Dinh’s convolutional neural network that performs object segmentation would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Dinh’s convolutional neural network that performs object segmentation in capacity management and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of ANBAI (US 20090240984 A1 Hereinafter “ANBAI”) in further view of Rohanna (US 4438831 A Hereinafter “Rohanna”) in further view of ZHANG (CN 111242004 A Hereinafter “ZHANG”).
Regarding claim 10, the combination of Lin, Strong, ANBAI, and Rohanna teaches the sensor module of claim 1, wherein the DNN is a convolutional neural network
The combination of Lin, Strong, ANBAI, and Rohanna does not expressly disclose where the neural network performs human pose estimation.
However, ZHANG teaches a neural network performs human pose estimation in elevators (Page 9, second to last paragraph: “the human posture of the elevator rider can be quickly detected by the human posture estimation neural network model”).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Lin, Strong, ANBAI, and Rohanna’s person detection system to include ZHANG’s neural network capable of human pose estimation because such a modification is taught, suggested, or motivated by the art. More specifically, the motivation to modify the combination of Lin, Strong, ANBAI, and Rohanna to include ZHANG is expressly provided by ZHANG, stating that their invention allows for the safety of the passengers to be determined quickly “whether the elevator rider is in an emergency state can be judged within 0.6 second, and the safety personnel can be directly informed to carry out first-time rescue arrangement by a background if the elevator is in the states of falling and waving for help.” (Page 9, second to last paragraph). Therefore, it would have been obvious to one of ordinary skill in the art at the time of the invention to modify the combination of Lin, Strong, ANBAI, and Rohanna’s person detection system to include ZHANG’s neural network capable of human pose estimation with the motivation of improving rider safety. The person of ordinary skill in the art would have recognized the benefit of improved rider safety.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of Chen (US 20190304102 A1 Hereinafter “Chen”).
Regarding claim 11, Lin teaches a computer-implemented method comprising:
capturing, by a camera, a first frame of a scene within the field of view (FOV) of the camera (Fig. 1B, [0022]: “As shown in FIG. 1B, additional video camera 32 located in elevator cab 18 provides video input with respect to the interior of elevator cab 18 to video processor 16”);
determining, based on the first frame, one or more detections indicative of the presence of one or more persons within the FOV of the camera (Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected);
while determining the one or more detections for the first frame, capturing, by the camera, a second frame of the scene within the FOV of the camera (Fig. 1B, [0022]: “As shown in FIG. 1B, additional video camera 32 located in elevator cab 18 provides video input with respect to the interior of elevator cab 18 to video processor 16”. Video input would contain multiple frames including a second frame);
after determining the one or more detections based on the first frame, determining based on the first frame and the one or more detections(Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected);
(Fig. 1B, [0022]: “As shown in FIG. 1B, additional video camera 32 located in elevator cab 18 provides video input with respect to the interior of elevator cab 18 to video processor 16”. Video input would contain multiple frames including a third frame since they are monitored from when they get on to when the get off which would encompass at least 3 frames “In scenarios in which some floors accessed by elevator cab 18 are secure, and other are not, then passengers must be monitored within elevator cab 18 to determine if an unauthorized user has gotten off on an authorized floor” [0065]); and
(Fig. 1A: The communication between the video processor and the control system).
Lin does not expressly disclose using a DNN for the passenger determination.
However, Strong teaches an ASIC programmable to implement a DNN for detecting the people ([00376]: “In the illustrated example, pixel-domain CNN 3400 performs object detection and classification for visual analytics using data in the pixel or image domain (e.g., using decompressed visual data)”. These object are said are said to be alternative with people “vision operations (e.g., person or object detection, facial recognition” [0160]. The implementations of these processes on processors are also said to be on ASICs “In various embodiments, object recognition processor 6613, semantic processor 6614, and inference processor 6615 may be implemented using any suitable combination of hardware and/or software logic, and may further be implemented as separate logical or physical components or combined into one or more integrated components. In some embodiments, for example, processors 6613-6615 may be implemented using general-purpose central processing units (CPUs), graphics processing units (GPUs), and/or special-purpose processors designed for artificial intelligence, machine learning, and/or neural network applications (e.g., using application-specific integrated circuits (ASICs) or field-programmable gate array (FPGAs)), among other examples” [0679]).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Lin’s passenger counting system to include Strong’s DNN on an ASIC for object detection because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Strong’s DNN on an ASIC for object detection permits high object detection accuracy ([0279]: These topologies require millions of images to converge with high accuracy). This known benefit in Strong is applicable to Lin’s passenger counting system as they both share characteristics and capabilities, namely, they are directed to detecting objects in images (frames are similar to images). Therefore, it would have been recognized that modifying Lin’s passenger counting system to include Strong’s DNN on an ASIC for object detection would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Strong’s DNN on an ASIC for object detection in detecting objects in images (frames are similar to images) and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Lin does not expressly disclose determining trackers for objects through multiple frames.
However, Chen teaches determining trackers for objects through multiple frames ([0067]: “As described in more detail herein, a video analytics system can generate and detect foreground blobs that can be used to perform various operations, such as object tracking (also called blob tracking) and/or the other operations described above. A blob tracker (also referred to as an object tracker) can be used to track one or more blobs in a video sequence using one or more bounding boxes”. It is said that these video analytics like blob tracking can be used for people counting “Video analytics can also act as an intrusion detector, a video counter (e.g., by counting people, objects, vehicles, or the like), a camera tamper detector, an object left detector, an object/asset removal detector, an asset protector, a loitering detector, and/or as a slip and fall detector” ([0066]). These tracker determinations happen over multiple frames “After being associated with a blob for a certain threshold duration (e.g., the threshold duration T2 described above), the split-new tracker can be transitioned to the normal status at a second frame. At the second frame at which the split-new tracker is transitioned to the normal status, a classification invocation request can be generated for the split-new tracker” [0149]).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Lin’s person detection system to include Chen’s person trackers because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Chen’s person trackers permits accurate detecting, tracking, and counting of detected individuals in the FOV of a video camera. This known benefit in Chen is applicable to the combination of Lin’s person detection system as they both share characteristics and capabilities, namely, they are directed to counting people in video. Therefore, it would have been recognized that modifying the combination of Lin’s person detection system to include Chen’s person trackers would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Chen’s person trackers in counting people in video and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of ANBAI (US 20090240984 A1 Hereinafter “ANBAI”) in further view of Rohanna (US 4438831 A Hereinafter “Rohanna”) in further view of Chen (US 20190304102 A1 Hereinafter “Chen”) in further view of Huang (US 20220188554 A1 Hereinafter “Huang”) in further view of WANG et al. (CN 109368434A Hereinafter “WANG”).
Regarding claim 5, the combination of Lin, Strong, ANBAI, Rohanna, Chen, and Huang teaches the sensor module of claim 4, in addition, Lin further teaches further comprising:
The combination of Lin, Strong, ANBAI, Rohanna, Chen, and Huang does not expressly disclose determining elevator capacity information from an acceleration sensor, which drives the GPIO from a low voltage state to high voltage state.
However, WANG teaches determining elevator capacity information from an acceleration sensor (Page 9, paragraph 8: “When the acceleration is greater than 0, the vehicle is in an overweight state; when the acceleration is less than 0, the device is in a weightlessness state”. Overweight state means the elevator is at capacity, and would result in the driving of the GPIO from low voltage to high voltage state).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Lin, Strong, ANBAI, Rohanna, Chen, and Huang’s elevator control system to include WANG’s acceleration sensor and determination of overweight status because such a modification is based on the use of known techniques to improve similar devices in the same way. More specifically, WANG’s determination of the elevator being overweight is comparable to the combination of Lin, Strong, ANBAI, Rohanna, Chen, and Huang’s imaging system to determine the elevator is at capacity because both methods determine if the elevator is full. Therefore, it would be obvious to one of ordinary skill in the art to modify the combination of Lin, Strong, ANBAI, Rohanna, Chen, and Huang’s elevator control system to include WANG’s acceleration sensor and determination of overweight status in order to obtain the predictable result of an additional method to verify if the elevator is at capacity (overweight).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of ANBAI (US 20090240984 A1 Hereinafter “ANBAI”) in further view of Rohanna (US 4438831 A Hereinafter “Rohanna”) in further view of Scoville (US 20170225921 A1 Hereinafter “Scoville”).
Regarding claim 6, the combination of Lin, Strong, ANBAI, and Rohanna teaches the sensor module of claim 1, in addition, Lin further teaches further comprising:
Lin does not expressly disclose driving from low to high voltage based on the detected people to put the elevator in a bypass mode.
However, Rohanna teaches driving from low to high voltage based on the detected people to put the elevator in a bypass mode (Col. 6, lines 30-35: “Relay BP includes contacts BP-2 in the hall call circuits 108, with the change in their condition when relay BP is energized, preventing the elevator car from answering hall calls”)
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Lin’s elevator system to include Rohanna’s hardware for activating bypass mode alongside the driving from low to high voltage because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Rohanna’s hardware for activating bypass mode alongside the driving from low to high voltage permits putting the elevator in bypass mode using defined hardware and activation requirements. This known benefit in Rohanna is applicable to Lin’s elevator system as they both share characteristics and capabilities, namely, they are directed to activation of a bypass mode in elevators. Therefore, it would have been recognized that modifying in’s elevator system to include Rohanna’s hardware for activating bypass mode alongside the driving from low to high voltage would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Rohanna’s hardware for activating bypass mode alongside the driving from low to high voltage in activation of a bypass mode in elevators and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Lin does not expressly disclose a general purpose input output (GPIO) for controlling the elevator.
However, ANBAI teaches a general purpose input output (GPIO) selectively controllable to output at least one of a low voltage state and a high voltage state ([0039]: “Also, the GPIO 220a closes the relay 210a when a value of "1" is written in channel ch1 by the I2C control unit 130a. On the other hand, the GPIO 220a opens the relay 210a when a value of "0" is written in channel ch1”).
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify Lin’s control system to include ANBAI’s GPIO capable of activating a relay because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, ANBAI’s GPIO capable of activating a relay permits a device for driving low voltage to high voltage (activating a relay). This known benefit in ANBAI is applicable to Lin’s control system as they both share characteristics and capabilities, namely, they are directed to activation of relays for electronic processes. Therefore, it would have been recognized that modifying Lin’s control system to include ANBAI’s GPIO capable of activating a relay would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate ANBAI’s GPIO capable of activating a relay in activation of relays for electronic processes and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
The combination of Lin, Strong, ANBAI, and Rohanna does not expressly disclose the use of a barometric pressure sensor (altimeter) for determining the altitude of the elevator.
However, Scoville teaches the use of a barometric pressure sensor (altimeter) for determining the altitude of the elevator ([0006]: “wherein the elevator control module is configured to determine a current altitude of the elevator car corresponding to a respective floor based on the measured first barometric pressure output from the at least one second barometric pressure sensor”. The driving of the GPIO from low voltage to high voltage is based off this determined altitude because the elevator’s altitude is taken into consideration when determining the floors that need to be bypassed and the floor that it is called to let passengers out.
At the time the invention was made, it would have been obvious to one of ordinary skill in the art to modify the combination of Lin, Strong, ANBAI, and Rohanna’s elevator floor bypass system to include Scoville barometric pressure sensor because such a modification is the result of applying a known technique to a known device ready for improvement to yield predictable results. More specifically, Scoville barometric pressure sensor permits accurate floor determination. This known benefit in Scoville barometric pressure sensor is applicable to the combination of Lin, Strong, ANBAI, and Rohanna’s elevator floor bypass system as they both share characteristics and capabilities, namely, they are directed to elevator control systems. Therefore, it would have been recognized that modifying the combination of Lin, Strong, ANBAI, and Rohanna’s elevator floor bypass system to include Scoville barometric pressure sensor would have yielded predictable results because (i) the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate Scoville barometric pressure sensor in elevator control systems and (ii) the benefits of such a combination would have been recognized by those of ordinary skill in the art.
Claims 12-15 are rejected under 35 U.S.C. 103 as being unpatentable over Lin et al. (US 20090057068 A1 Hereinafter “Lin”) in view of Strong et al. (US 20190043201 A1 Hereinafter “Strong”) in further view of WANG et al. (CN 109368434A Hereinafter “WANG”) in further view of Scoville (US 20170225921 A1 Hereinafter “Scoville”).
Regarding claim 12, Lin teaches a system comprising:
a camera module comprising an image processor and a lens that collectively capture image data representative of a field of view (FOV) a scene (Fig. 1B, [0022]: “As shown in FIG. 1B, additional video camera 32 located in elevator cab 18 provides video input with respect to the interior of elevator cab 18 to video processor 16”. The video camera has a lens and processor to take video, and these work together to capture images of a FOV of a scene (cab). The video captures a sequence of frames similar to images and the processor processes there frames similar to how an image processor processes images);
a non-transitory storage medium storing instructions thereon that, upon execution by the at least one processor (Fig. 1B, [0019]: “Video processor 16 provides passenger data to control system 24, providing control system 24 with additional information regarding elevator passengers”. The instructions for processing the video on the processor would be stored on a non-transitory storage medium storing instructions), performs operations comprising:
capturing, by the camera module, image data of the FOV of the camera module (Fig. 1B, [0022]: “As shown in FIG. 1B, additional video camera 32 located in elevator cab 18 provides video input with respect to the interior of elevator cab 18 to video processor 16”);
detecting, based on the captured image data, the presence of one or more persons within the FOV of the camera (Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected);
counting, by the at least one processor, a number of persons detected within the FOV of the camera (Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage parameters in elevator cab 18, as well as the available elevator cab area for additional passengers”. If the number of passengers are determined they are detected); and
(Fig. 1B, [0022]: “For instance, video processor 16 determines the number of passengers or other usage para