DETAILED ACTION
This Action is in response to Applicant’s response filed on 03/18/2026. Claims 1-9, 18-19 and newly adding claim 20 are still pending in the present application. Claims 10-17 are canceled. This Action is made FINAL.
Response to Amendment
Drawing Objection: The amended claims filed on 03/18/2026 overcomes the Drawing Objection in the previous office action.
Response to Arguments
Applicant's arguments filed on 03/18/2026 have been fully considered but are moot in view of the new ground(s) rejection in view of Ali et al (U.S. 11,308,316 B1; Ali).
Claim Status
Claim(s) 1-9 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Miyamoto “Passenger in vehicle counting method of HOV/HOT system”, in view of Ali et al (U.S. 11,308,316 B1; Ali).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-9 and 18-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Miyamoto “Passenger in vehicle counting method of HOV/HOT system”, in view of Ali et al (U.S. 11,308,316 B1; Ali).
Regarding claim 1, Miyamoto discloses a computer implemented method for determining a number of persons in a vehicle, (Abstract: “a novel number of passengers counting method including system architecture, its image processing context and experiment result is described”) comprising:
obtaining multiple images of a recording area in chronological succession from a camera; (Fig.1 and I. Introduction: “The proposed system is composed of four main equipment: floodlights, cameras, a laser sensor, and processing unit (Fig. 1 – upper column). To count the number of passengers not only in the front row of sheets but also in rear rows, the camera is basically installed on the roadside and it captures side view of a vehicle.”; IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)). When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)),” )
storing the multiple images in a data memory; (IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)). When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)).”)
obtaining a known distance between the vehicle and the recording area; (Fig.1 and I. INTRODUCTION : “the distance between camera and vehicle has no chance but to be short. two cameras are used, one of which is fixed at a higher position of camera supporting pole and another one is fixed at a lower position. After operating image processing on those two image streams, appropriate result of those will be adopted.”)
obtaining a position of the vehicle;( (B. Integration of face detection result: “This integration module is composed of three subroutines, the amount estimation of vehicle movement, In the first subroutine, we estimate the vehicle moving amount based on time-sequential image difference. If we denote the pixel value at the position p on time=T image and time= T-1 image as It(p) and It-1(p) respectively”)
determining a velocity of the vehicle based on the position; (B. Integration of face detection result: “This integration module is composed of three subroutines, the amount estimation of vehicle movement, In the first subroutine, we estimate the vehicle moving amount based on time-sequential image difference. If we denote the pixel value at the position p on time=T image and time= T-1 image as It(p) and It-1(p) respectively, the vehicle moving amount Tx will be estimated as follows: equation (1) …. These images have similar vehicle moving amount, and we can make an assumption that Xt is same among different T.”)
determining an arrival point in time of the vehicle in the recording area based on the known distance, the position, and the velocity; (IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)). When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)). If a prominent motion of a vehicle is observed in the stored stream, still image at that time is dispatched to passenger counting process (Fig. 2(d)) … total image processing throughput is kept in nearly constant even if the speed of target vehicle has the wide range from 0km/h to 200km/h”)
selecting at least one image of the multiple images from the data memory based on the arrival point in time, wherein the at least one image was recorded in a time range including the arrival point in time; (IV. Data Flow – A. Image dispatching process: “When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)). If a prominent motion of a vehicle is observed in the stored stream, still image at that time is dispatched to passenger counting process (Fig. 2(d)) … In case of traffic congestion, one vehicle may keep stopping in front of the roadside camera and camera stream will be consisted of redundant images. Image dispatching process plays a role to select meaning images among these redundant ones.”; IV. DATA FLOW- B. Threads and GPU After image dispatching process, images to be used in following passenger counting process are selected from camera stream … Basically, one thread is assigned to one selected image and both of preprocessing and face detection is executed by using this thread (Fig. 3).”) and
determining the number of persons in the vehicle based on the selected at least one image. (Fig.5 and IV. DATA FLOW- B. Threads and GPU After image dispatching process, images to be used in following passenger counting process are selected from camera stream” ; V. PASSENGER COUNTING PROCESS: “The image processing function for the number of passengers counting consists of three components: face detection, detected face integration, and confidence value calculation. From each still frame, all people’s faces are detected one by one. By combining plural detection result of the same passenger observed in several time-sequential frames as one passenger, we can finally decide the number of passengers in the vehicle.”)
However, Miyamoto does not disclose retrieving a known distance between the vehicle and the recording area;
Ali discloses a computer implemented method for determining a number of persons in a vehicle, (Col 5- lines 45-47: “computer vision based Vehicle Occupancy Detection (VOD) systems and methods.”) comprising:
obtaining multiple images of a recording area in chronological succession from a camera; (Col 5- lines 47-52: “a road-side unit (iRSU) with an imaging device that captures successive images of vehicles moving along in a lane on a highway or road. The road-side unit is configured to capture a plurality of images of one or more vehicles travelling along a lane of a road.”)
storing the multiple images in a data memory; (Col 28- lines 49-52: “the computing device 102 may store the received images into database 112, including a timestamp and metadata (number of people, debugging data, camera parameters, LIDAR triggering information, etc.).”)
retrieving a known distance between the vehicle and the recording area; (Col 8- lines 42-67: “vehicle detector(s) 114 can be various devices which are capable of detecting the presence of a vehicle at various distances. … a LiDAR vehicle detector(s) 114 can be configured to ignore any readings representative of objects more than 10 m away. In some embodiments, for example, where the vehicle detector(s) 114 include multiple devices, each of the multiple devices can be configured to detect vehicles at different distances. the multiple devices may be used redundantly to detect vehicles a single distance away from the vehicle detector(s)”; Col 10 -lines 58 to Col 11- lines 1-2: “The vehicle detector controller 104 can be configured to control the vehicle detector(s) 114 through a series of command signals. … the vehicle detector(s) 114 may be configured to expect vehicles at the detection distance to be near or close to the top of a field of view of the vehicle detector(s) 114.”; Col 21 -lines 12-20)
obtaining a position of the vehicle; (Col 17 – lines 1-15 : “the vehicle detector may be upstream of the imaging device if the vehicle detector produces only a one-dimensional lateral measurement of the distance between the station and the vehicle. … Such sensors can be placed in different locations as the physical and geometrical placement plays a less important role, relative to the role of the 3D perception process that detects and tracks the vehicle, and triggers images by the imaging device.”; Col 21 -lines 12-20: “a plurality of vehicle detectors 114, each vehicle detector may be respectively positioned to detect vehicles at different distances.”; Col 22 – lines 15-19: “the vehicle 604 is detected, the vehicle position, direction of travel and speed is tracked using a tracking approach, such as a Kalman filter.”)
determining a velocity of the vehicle based on the position; (Col 11 – lines 3-13: “the vehicle detector controller 104 may transmit command signal to the vehicle detector(s) 114 to detect the speed of a vehicle. The vehicle detector(s) 114 may, in response, provide the vehicle detector controller 104 with two detections of the same car at different instances in time, allowing for the speed to be interpolated. … the vehicle detector(s) 114 is continuously monitoring detected vehicles in its field of view, and directly computes the speed of the detected vehicles and relays the same to the vehicle detector controller 104.”; Col 22 – lines 15-19 : “the vehicle 604 is detected, the vehicle position, direction of travel and speed is tracked using a tracking approach, such as a Kalman filter.)
determining an arrival point in time of the vehicle in the recording area based on the known distance, the position, and the velocity; (Col 21 – lines 21-43: “The vehicle detectors may be 2D or 3D LIDAR units capable of capturing multiple readings of distances in multiple directions. … ice, the more robust detection of passing vehicles may provide for more precise adjustments of the pattern between the light emission, image capture and vehicle detection. In example embodiments, the more precise estimation allows for detecting the vehicles a greater distance from the system 702, and allows greater filtering windows (i.e., the use of larger windows of time between detection of the vehicle and capturing an image of the vehicle) without risking detecting the car too late.”; Col 22 – lines 15-24: “once the vehicle 604 is detected, the vehicle position, direction of travel and speed is tracked using a tracking approach, such as a Kalman filter. The estimation of position and speed of the vehicle 604 can then be used to trigger, for example, the license imaging device(s) 118-3 of FIG. 7 (e.g., when the vehicle 604 is about 10-14 meters upstream of system 702), and then trigger the imaging devices 118-1 and 118-2 multiple times at optimal places to take multiple shots for occupancy counting.”)
determining the number of persons in the vehicle based on the selected at least one image. (Col 11 – lines 57-63: “The occupant detector 110 is configured to receive the plurality of images from the imaging device(s) 118 (and possibly second imaging device 126) and determine a number of occupants in the detected vehicle (alternatively referred to as a vehicle occupancy). The occupant detector 110 may be a machine learning model trained to determine the vehicle occupancy based on roadside images.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Miyamoto by including Vehicle detector(s) and a vehicle detector controller that is taught by Ali, to make the invention that a system for detecting occupancy of a vehicle travelling in a direction of travel along a road.; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving automated vehicle occupancy detection systems which are more accurate, faster, more reliable, easier to move or install, more robust, or require less calibration are desirable.. (Ali: Paragraph 4)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 2, Miyamoto, as modified by Ali, discloses all the claims invention. Miyamoto further discloses further comprising illuminating the recording area with a light source at recording points in time, (Fig.1 and III. HARDWARE – A. Flood light: “ the proposed system adopted large intensity floodlight. On the viewpoint of preventing from blinding driver’s eye in the front camera system, the wavelength of flood light is about 850nm. On the other hand, inside camera system, floodlight with the wavelength of 730nm is used because of low risk of blinding driver’s eye. To make light intensity strong as possible, lighting illuminates in flush and its frequency is equivalent to the frame rate of the camera.”) wherein the camera captures the multiple images of the recording area at the recording points in time. (IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)).”; V. PASSENGER COUNTING PROCESS- B. Integration of face detection result: “we estimate the vehicle moving amount based on time-sequential image difference. If we denote the pixel value at the position p on time =T image and time= T −1 image as IT (p) and I [T-1] (p), respectively,”)
Regarding claim 3 Miyamoto, as modified by Ali, discloses all the claims invention. Miyamoto further discloses wherein the light source emits electromagnetic radiation at a wavelength between 720 nm and 750 nm. (Fig.1 and III. HARDWARE – A. Flood light: “ the proposed system adopted large intensity floodlight. On the viewpoint of preventing from blinding driver’s eye in the front camera system, the wavelength of flood light is about 850nm. On the other hand, inside camera system, floodlight with the wavelength of 730nm is used because of low risk of blinding driver’s eye.”)
Regarding claim 4, Miyamoto, as modified by Ali, discloses all the claims invention. Miyamoto further discloses wherein the light source emits electromagnetic radiation at least one of: 1) having a radiant power of at least 1,000,000 lumen; or 2) in a pulsed manner, having an illumination time of less than 70 ps, at the recording points in time. (III. HARDWARE - A: Floodlight: “To count the number of passengers through these glasses, the proposed system adopted large intensity floodlight … the wavelength of flood light is about 850nm. On the other hand, inside camera system, floodlight with the wavelength of 730nm is used”; B. Camera : “Corresponding to the wavelength of floodlight, effective wavelength range of adopted camera covers from 300nm to 900nm.”, the person ordinary skill in the art would be know that the wavelength in range of “visible light” have a radiant power of about 1 Watt which correspond to emits around 1,000,000 lumen of visible light).
Regarding claim 5, Miyamoto, as modified by Ali, discloses all the claims invention. Miyamoto further discloses wherein the camera records images at a recording rate of more than 40 images per second. (III.HARDWARE - B. Camera: “a camera with 100 fps is chosen in proposed system to satisfy this requirement.”)
Regarding claim 6, Miyamoto, as modified by Ali, discloses all the claims invention. Miyamoto further discloses selecting the at least one image comprises selecting a plurality of images of the multiple images from the data memory based on the arrival point in time, wherein the plurality of images were recorded in the time range, (IV. Data Flow – A. Image dispatching process: “When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)). If a prominent motion of a vehicle is observed in the stored stream, still image at that time is dispatched to passenger counting process (Fig. 2(d)) … In case of traffic congestion, one vehicle may keep stopping in front of the roadside camera and camera stream will be consisted of redundant images. Image dispatching process plays a role to select meaning images among these redundant ones.”; IV. DATA FLOW- B. Threads and GPU After image dispatching process, images to be used in following passenger counting process are selected from camera stream”) and wherein determining the number of persons in the vehicle comprises: determining a respective candidate number of persons in the vehicle for each of the plurality of images; determining which of the plurality of images shows a greatest number of persons in the vehicle among the plurality of images based on the determined respective candidate number of persons in the vehicle for each of the plurality of images; and determining the greatest number of persons as the number of persons in the vehicle. (Figs. 5, 6, 7; and V. PASSENGER COUNTING PROCESS: “B. Integration of face detection result: “This integration module is composed of three subroutines, the amount estimation of vehicle movement, face detection result classification, and error rejection … we link face detection results over different time. Assume that at time=T, there are two face detection results, D1 (T) and D2 (T) , and the previous one is the right side of later one. As same, at time=T+1 there are three detection results Di (T + 1) (i = 1,...,3) and they are aligning right to left in the image with upper suffix ascending order (Fig. 6). … integration process can suppress grouping fault of face detection results.”; VI. EXPERIMENT - B. Passenger counting accuracy – Table 1-2-3)
Regarding claim 7, Miyamoto, as modified by Ali, discloses all the claims invention. Miyamoto further discloses determining the number of persons in the vehicle comprises evaluating the at least one image with an artificial intelligence model to recognize how many persons are in the vehicle in the at least one image. (V. PASSENGER COUNTING PROCESS- A. Face detection from still image: “The precise face detection determines the system accuracy upper limitation and so it is very important what detection strategy we choose. Among recent progressing deep learning techniques [6][7][8], we just applied VGG16 deep CNN [9] to our system because it realizes high performance in many detection and classification problem (Fig. 5). Among objects inside a vehicle, a headrest is often detected as face pattern when the image contrast inside the vehicle is not good. Because of this reason, our face detection CNN has three output unit corresponding to “face”, “not face” and “headrest”.”; VI. EXPERIMENT-A. Precision and recall of face detection: “Fig. 8 shows the validation result in terms of Precision-Recall curve with 850 of facial images.”)
Regarding claim 8, Miyamoto, as modified by Ali, discloses all the claims invention. Miyamoto further discloses selecting the at least one image comprises selecting a plurality of images of the multiple images from the data memory based on the arrival point in time, wherein the plurality of images were recorded in the time range, (IV. Data Flow – A. Image dispatching process: “When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)). If a prominent motion of a vehicle is observed in the stored stream, still image at that time is dispatched to passenger counting process (Fig. 2(d)) … In case of traffic congestion, one vehicle may keep stopping in front of the roadside camera and camera stream will be consisted of redundant images. Image dispatching process plays a role to select meaning images among these redundant ones.”; IV. DATA FLOW- B. Threads and GPU After image dispatching process, images to be used in following passenger counting process are selected from camera stream”)and wherein determining the number of persons in the vehicle comprises: evaluating the plurality of images with an artificial intelligence model to determine a respective candidate number of persons in the vehicle for each of the plurality of images; (V. PASSENGER COUNTING PROCESS- A. Face detection from still image: “The precise face detection determines the system accuracy upper limitation and so it is very important what detection strategy we choose. Among recent progressing deep learning techniques [6][7][8], we just applied VGG16 deep CNN [9] to our system because it realizes high performance in many detection and classification problem (Fig. 5). Among objects inside a vehicle, a headrest is often detected as face pattern when the image contrast inside the vehicle is not good. Because of this reason, our face detection CNN has three output unit corresponding to “face”, “not face” and “headrest”.”; VI. EXPERIMENT-A. Precision and recall of face detection: “Fig. 8 shows the validation result in terms of Precision-Recall curve with 850 of facial images.”)
determining which of the plurality of images shows a greatest number of persons in the vehicle among the plurality of images based on the determined respective candidate number of persons in the vehicle for each of the plurality of images; and determining the greatest number of persons as the number of persons in the vehicle. (Figs. 5, 6, 7; and V. PASSENGER COUNTING PROCESS: “B. Integration of face detection result: “This integration module is composed of three subroutines, the amount estimation of vehicle movement, face detection result classification, and error rejection … we link face detection results over different time. Assume that at time=T, there are two face detection results, D1 (T) and D2 (T) , and the previous one is the right side of later one. As same, at time=T+1 there are three detection results Di (T + 1) (i = 1,...,3) and they are aligning right to left in the image with upper suffix ascending order (Fig. 6). … integration process can suppress grouping fault of face detection results.”; VI. EXPERIMENT - B. Passenger counting accuracy – Table 1-2-3)
Regarding claim 9, Miyamoto, as modified by Ali discloses further comprising: obtaining additional images of a second recording area in chronological succession from a second camera; (Miyamoto: Fig.1 show the near lane, far lane and front camera system and I. INTRODUCTION: “ The proposed system is composed of four main equipment: floodlights, cameras, a laser sensor, and processing unit (Fig. 1– upper column). … In such case, one camera will be installed at the gantry for capturing front sheet image and another camera will be installed at the roadside for shooting sheets except for front ones (Fig. 1 – lower column).”)
storing the additional images in the data memory; (Miyamoto:Fig.1 shows the near lane, far lane and front camera system; IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)). When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)).”)
obtaining a second position of the vehicle; (Ali: Fig.6A -6D; Col 21 -lines 12-20: “the vehicle detector(s) 114 may detect vehicles approximately 15 to 20 meters before they are in the field of view of the respective imaging devices. In example embodiments including a plurality of vehicle detectors 114, each vehicle detector may be respectively positioned to detect vehicles at different distances.”; Col 18-lines 48-64: “Both the imaging device(s) 118 and the vehicle detector(s) 114 are located a distance 618 from an expected position of the vehicle … the distance from the imaging device(s) 118 to the expected position of region of interest 614 (e.g., the distance from the imaging device(s) 118 to the middle of the lane) may be different from the distance from the vehicle detector(s) 114 to the expected position of region of interest 614 (e.g., the distance from the vehicle detector(s) 114 to the middle of the lane).”)
determining a second velocity of the vehicle based on the second position; (Ali: Col 18 -lines 11-17: “ The yaw angle may increase the accuracy of the vehicle detection system by forcing images to contain certain perspectives, which, when multiple consecutive images are captured of vehicle 604 traveling at different speeds are captured at said perspectives, contributes to all front and rear occupants of the vehicle 604 being visible in the captured images.”; Col 23 – lines 1-9: “at higher detected vehicle speeds, the imaging device(s) 118 may capture images at a greater frequency (i.e., a higher FPS) or imaging device(s) 118 may begin capturing images more rapidly in response to a vehicle being detected (e.g., using a shorter filtering algorithm window). Conversely, at lower detected vehicle speeds, imaging device(s) 118 may reduce the frequency of image capture (i.e., the FPS) or increase the algorithm window size”)
determining a second arrival point in time of the vehicle in the second recording area based on the known distance, the second position, and the second velocity (Miyamoto:Fig.1 shows the near lane, far lane and front camera system; I. Introduction: “two cameras are used, one of which is fixed at a higher position of camera supporting pole and another one is fixed at a lower position. After operating image processing on those two image streams, appropriate result of those will be adopted.”; (IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)). When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)). If a prominent motion of a vehicle is observed in the stored stream, still image at that time is dispatched to passenger counting process (Fig. 2(d)) … total image processing throughput is kept in nearly constant even if the speed of target vehicle has the wide range from 0km/h to 200km/h”); B. Passenger counting accuracy: “Estimation accuracy for near lane is summarized in TABLE 1 and the one for the far lane is in TABLE 2 … ) and
selecting at least one additional image of the additional images from the data memory based on the second arrival point in time, wherein the at least one additional image was recorded in a second time range including the second arrival point in time, (Miyamoto:Fig.1 show the for near lane, far lane and front camera system; IV. Data Flow – A. Image dispatching process: “When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)). If a prominent motion of a vehicle is observed in the stored stream, still image at that time is dispatched to passenger counting process (Fig. 2(d)) … In case of traffic congestion, one vehicle may keep stopping in front of the roadside camera and camera stream will be consisted of redundant images. Image dispatching process plays a role to select meaning images among these redundant ones.”; IV. DATA FLOW- B. Threads and GPU After image dispatching process, images to be used in following passenger counting process are selected from camera stream …Basically, one thread is assigned to one selected image and both of preprocessing and face detection is executed by using this thread (Fig. 3). ”)
wherein determining the number of persons in the vehicle based on the selected at least one image (Miyamoto: Fig.5 and IV. DATA FLOW- B. Threads and GPU After image dispatching process, images to be used in following passenger counting process are selected from camera stream” ; V. PASSENGER COUNTING PROCESS: “The image processing function for the number of passengers counting consists of three components: face detection, detected face integration, and confidence value calculation. From each still frame, all people’s faces are detected one by one. By combining plural detection result of the same passenger observed in several time-sequential frames as one passenger, we can finally decide the number of passengers in the vehicle.”) comprises: determining a first candidate number of persons in the vehicle based on the selected at least one image; determining a second candidate number of persons in the vehicle based on the selected at least one additional image; and determining the number of persons in the vehicle based on a greater of the first candidate number of persons and the second candidate number of persons. (Miyamoto: Figs. 5, 6, 7; and V. PASSENGER COUNTING PROCESS: “B. Integration of face detection result: “This integration module is composed of three subroutines, the amount estimation of vehicle movement, face detection result classification, and error rejection … we link face detection results over different time. Assume that at time=T, there are two face detection results, D1 (T) and D2 (T) , and the previous one is the right side of later one. As same, at time=T+1 there are three detection results Di (T + 1) (i = 1,...,3) and they are aligning right to left in the image with upper suffix ascending order (Fig. 6). … integration process can suppress grouping fault of face detection results.”; VI. EXPERIMENT - B. Passenger counting accuracy – Table 1-2-3)
Regarding claim 18, Miyamoto discloses a method for determining a number of persons in a vehicle, (Abstract: “a novel number of passengers counting method including system architecture, its image processing context and experiment result is described”) the method comprising:
obtaining multiple images of a recording area in chronological succession from a camera; (Fig.1 and I. Introduction: “The proposed system is composed of four main equipment: floodlights, cameras, a laser sensor, and processing unit (Fig. 1 – upper column). To count the number of passengers not only in the front row of sheets but also in rear rows, the camera is basically installed on the roadside and it captures side view of a vehicle.”; IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)). When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b))”)
illuminating the recording area with a light source at recording points in time, (Fig.1 and III. HARDWARE – A. Flood light: “ the proposed system adopted large intensity floodlight. On the viewpoint of preventing from blinding driver’s eye in the front camera system, the wavelength of flood light is about 850nm. On the other hand, inside camera system, floodlight with the wavelength of 730nm is used because of low risk of blinding driver’s eye. To make light intensity strong as possible, lighting illuminates in flush and its frequency is equivalent to the frame rate of the camera.”)
wherein the camera captures the multiple images of the recording area at the recording points in time, (IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)).”; V. PASSENGER COUNTING PROCESS- B. Integration of face detection result: “we estimate the vehicle moving amount based on time-sequential image difference. If we denote the pixel value at the position p on time =T image and time= T −1 image as IT (p) and I [T-1] (p), respectively,”)
wherein the light source emits electromagnetic radiation at a wavelength between 720 nm and 750 nm, (Fig.1 and III. HARDWARE – A. Flood light: “On the other hand, inside camera system, floodlight with the wavelength of 730nm is used because of low risk of blinding driver’s eye. To make light intensity strong as possible, lighting illuminates in flush and its frequency is equivalent to the frame rate of the camera.”) and wherein the camera records images at a recording rate of more than 40 images per second; (III.HARDWARE – B. Camera: “a camera with 100 fps is chosen in proposed system to satisfy this requirement.”)
storing the multiple images in a data memory; (IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)). When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)).”)
obtaining a position of the vehicle;( (B. Integration of face detection result: “This integration module is composed of three subroutines, the amount estimation of vehicle movement, In the first subroutine, we estimate the vehicle moving amount based on time-sequential image difference. If we denote the pixel value at the position p on time=T image and time= T-1 image as It(p) and It-1(p) respectively”)
determining a velocity of the vehicle based on the position; (B. Integration of face detection result: “This integration module is composed of three subroutines, the amount estimation of vehicle movement, In the first subroutine, we estimate the vehicle moving amount based on time-sequential image difference. If we denote the pixel value at the position p on time=T image and time= T-1 image as It(p) and It-1(p) respectively, the vehicle moving amount Tx will be estimated as follows: equation (1) …. These images have similar vehicle moving amount, and we can make an assumption that Xt is same among different T.”)
obtaining a known distance between the vehicle and the recording area; (Fig.1 and I. INTRODUCTION : “the distance between camera and vehicle has no chance but to be short. two cameras are used, one of which is fixed at a higher position of camera supporting pole and another one is fixed at a lower position. After operating image processing on those two image streams, appropriate result of those will be adopted.”)
determining an arrival point in time of the vehicle in the recording area based on the position, and the velocity and the known distance; (IV. Data Flow – A. Image dispatching process: “The processing unit receives image streams from camera continuously (Fig. 2 (a)). When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)). If a prominent motion of a vehicle is observed in the stored stream, still image at that time is dispatched to passenger counting process (Fig. 2(d)) … total image processing throughput is kept in nearly constant even if the speed of target vehicle has the wide range from 0km/h to 200km/h”)
selecting a plurality of images of the multiple images from the data memory based on the arrival point in time, wherein the plurality of images are recorded in a time range including the arrival point in time; (IV. Data Flow – A. Image dispatching process: “When the processing unit receives the trigger signal from laser sensor (Fig. 2 (b)), image dispatching process starts to store image stream in the buffer memory (Fig. 2(c)). If a prominent motion of a vehicle is observed in the stored stream, still image at that time is dispatched to passenger counting process (Fig. 2(d)) … In case of traffic congestion, one vehicle may keep stopping in front of the roadside camera and camera stream will be consisted of redundant images. Image dispatching process plays a role to select meaning images among these redundant ones.”; IV. DATA FLOW- B. Threads and GPU: “After image dispatching process, images to be used in following passenger counting process are selected from camera stream … Basically, one thread is assigned to one selected image and both of preprocessing and face detection is executed by using this thread (Fig. 3).”)
determining a respective candidate number of persons in the vehicle for each of the plurality of images; (Fig.5 and IV. DATA FLOW- B. Threads and GPU After image dispatching process, images to be used in following passenger counting process are selected from camera stream” ; V. PASSENGER COUNTING PROCESS: “The image processing function for the number of passengers counting consists of three components: face detection, detected face integration, and confidence value calculation. From each still frame, all people’s faces are detected one by one. By combining plural detection result of the same passenger observed in several time-sequential frames as one passenger, we can finally decide the number of passengers in the vehicle.”)
determining which of the plurality of images shows a greatest number of persons in the vehicle among the plurality of images based on the determined respective candidate number of persons in the vehicle for each of the plurality of images; and determining the greatest number of persons as a number of persons in the vehicle. (Figs. 5, 6, 7; and V. PASSENGER COUNTING PROCESS: “B. Integration of face detection result: “This integration module is composed of three subroutines, the amount estimation of vehicle movement, face detection result classification, and error rejection … we link face detection results over different time. Assume that at time=T, there are two face detection results, D1 (T) and D2 (T) , and the previous one is the right side of later one. As same, at time=T+1 there are three detection results Di (T + 1) (i = 1,...,3) and they are aligning right to left in the image with upper suffix ascending order (Fig. 6). … integration process can suppress grouping fault of face detection results.”; VI. EXPERIMENT - B. Passenger counting accuracy – Table 1-2-3)
However, Miyamoto does not disclose retrieving a known distance between the vehicle and the recording area;
Ali discloses a computer implemented method for determining a number of persons in a vehicle, (Col 5- lines 45-47: “computer vision based Vehicle Occupancy Detection (VOD) systems and methods.”) comprising:
obtaining multiple images of a recording area in chronological succession from a camera; (Col 5- lines 47-52: “a road-side unit (iRSU) with an imaging device that captures successive images of vehicles moving along in a lane on a highway or road. The road-side unit is configured to capture a plurality of images of one or more vehicles travelling along a lane of a road.”)
storing the multiple images in a data memory; (Col 28- lines 49-52: “the computing device 102 may store the received images into database 112, including a timestamp and metadata (number of people, debugging data, camera parameters, LIDAR triggering information, etc.).”)
obtaining a position of the vehicle; (Col 17 – lines 1-15 : “the vehicle detector may be upstream of the imaging device if the vehicle detector produces only a one-dimensional lateral measurement of the distance between the station and the vehicle. … Such sensors can be placed in different locations as the physical and geometrical placement plays a less important role, relative to the role of the 3D perception process that detects and tracks the vehicle, and triggers images by the imaging device.”; Col 21 -lines 12-20: “a plurality of vehicle detectors 114, each vehicle detector may be respectively positioned to detect vehicles at different distances.”; Col 22 – lines 15-19: “the vehicle 604 is detected, the vehicle position, direction of travel and speed is tracked using a tracking approach, such as a Kalman filter.”)
determining a velocity of the vehicle based on the position; (Col 11 – lines 3-13: “the vehicle detector controller 104 may transmit command signal to the vehicle detector(s) 114 to detect the speed of a vehicle. The vehicle detector(s) 114 may, in response, provide the vehicle detector controller 104 with two detections of the same car at different instances in time, allowing for the speed to be interpolated. … the vehicle detector(s) 114 is continuously monitoring detected vehicles in its field of view, and directly computes the speed of the detected vehicles and relays the same to the vehicle detector controller 104.”; Col 22 – lines 15-19 : “the vehicle 604 is detected, the vehicle position, direction of travel and speed is tracked using a tracking approach, such as a Kalman filter.)
retrieving a known distance between the vehicle and the recording area; (Col 8- lines 42-67: “vehicle detector(s) 114 can be various devices which are capable of detecting the presence of a vehicle at various distances. … a LiDAR vehicle detector(s) 114 can be configured to ignore any readings representative of objects more than 10 m away. In some embodiments, for example, where the vehicle detector(s) 114 include multiple devices, each of the multiple devices can be configured to detect vehicles at different distances. the multiple devices may be used redundantly to detect vehicles a single distance away from the vehicle detector(s)”; Col 10 -lines 58 to Col 11- lines 1-2: “The vehicle detector controller 104 can be configured to control the vehicle detector(s) 114 through a series of command signals. … the vehicle detector(s) 114 may be configured to expect vehicles at the detection distance to be near or close to the top of a field of view of the vehicle detector(s) 114.”; Col 21 -lines 12-20)
determining an arrival point in time of the vehicle in the recording area based on the position, and the velocity, and the known distance; (Col 21 – lines 21-43: “The vehicle detectors may be 2D or 3D LIDAR units capable of capturing multiple readings of distances in multiple directions. … ice, the more robust detection of passing vehicles may provide for more precise adjustments of the pattern between the light emission, image capture and vehicle detection. In example embodiments, the more precise estimation allows for detecting the vehicles a greater distance from the system 702, and allows greater filtering windows (i.e., the use of larger windows of time between detection of the vehicle and capturing an image of the vehicle) without risking detecting the car too late.”; Col 22 – lines 15-24: “once the vehicle 604 is detected, the vehicle position, direction of travel and speed is tracked using a tracking approach, such as a Kalman filter. The estimation of position and speed of the vehicle 604 can then be used to trigger, for example, the license imaging device(s) 118-3 of FIG. 7 (e.g., when the vehicle 604 is about 10-14 meters upstream of system 702), and then trigger the imaging devices 118-1 and 118-2 multiple times at optimal places to take multiple shots for occupancy counting.”)
determining the number of persons in the vehicle based on the selected at least one image. (Col 11 – lines 57-63: “The occupant detector 110 is configured to receive the plurality of images from the imaging device(s) 118 (and possibly second imaging device 126) and determine a number of occupants in the detected vehicle (alternatively referred to as a vehicle occupancy). The occupant detector 110 may be a machine learning model trained to determine the vehicle occupancy based on roadside images.”)
Therefore, it would been obvious to one having ordinary skill in the art before the effective filling date of the claimed invention to modify the invention of Miyamoto by including Vehicle detector(s) and a vehicle detector controller that is taught by Ali, to make the invention that a system for detecting occupancy of a vehicle travelling in a direction of travel along a road.; thus, one of ordinary skilled in the art would have been motivated to combine the references since this will improving automated vehicle occupancy detection systems which are more accurate, faster, more reliable, easier to move or install, more robust, or require less calibration are desirable.. (Ali: Paragraph 4)
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filling date of the claimed invention.
Regarding claim 19, Miyamoto, as modified by Ali, discloses all the claims invention. Miyamoto further discloses the light source emits electromagnetic radiation at least one of: 1) having a radiant power of at least 1,000,000 lumen; or 2) in a pulsed manner, having an illumination time of less than 70 ps, at the recording points in time. (III. HARDWARE - A: Floodlight: “To count the number of passengers through these glasses, the proposed system adopted large intensity floodlight … the wavelength of flood light is about 850nm. On the other hand, inside camera system, floodlight with the wavelength of 730nm is used”; B. Camera : “Corresponding to the wavelength of floodlight, effective wavelength range of adopted camera covers from 300nm to 900nm.”, the person ordinary skill in the art would be know that the wavelength in range of “visible light” have a radiant power of about 1 Watt which correspond to emits around 1,000,000 lumen of visible light).
Regarding claim 20, Miyamoto, as modified by Ali, discloses all the claims invention. Ali further discloses determining the number of persons in the vehicle comprises evaluating the plurality of images with an artificial intelligence model to determine the respective candidate number of persons in the vehicle for each of the plurality of images. (Col 11 – lines 57-63: “The occupant detector 110 is configured to receive the plurality of images from the imaging device(s) 118 (and possibly second imaging device 126) and determine a number of occupants in the detected vehicle (alternatively referred to as a vehicle occupancy). The occupant detector 110 may be a machine learning model trained to determine the vehicle occupancy based on roadside images.”)
Relevant Prior Art Directed to State of Art
Dalal et al (U.S. 20130141574 A1), Vehicle Occupancy Detection Via Single Band Infrared Imaging”, teaches about systems and methods for vehicle occupancy detection which utilize a single band infrared (IR) imaging system operating at a pre-defined wavelength range of the electromagnetic spectrum to capture an infrared image of a motor vehicle traveling in a restricted lane of traffic. It also teaches about determining a threshold reflectance value which is used, to isolate pixels in the image which are categorized as human skin from pixels of other materials detected in the vehicle's interior and then determine the number of human occupants in the vehicle's passenger compartment.
Shimizu (U.S. 20180338225 A1), “Human Occupancy Detection In a Transportation Vehicle”, teaches about system and method to determine a number of occupants in a transportation vehicle may include receiving a first indication of a first location, a first speed, a first acceleration, and a first time associated with a first device, and a second indication of a second location, a second speed, a second acceleration, and a second time associated with a second device.
Hayashi et al (U.S. 20200210732 A1), “Analysis Apparatus, Analysis Method, and Non-Transitory Storage Medium”, teaches about an analysis apparatus including a first image analysis unit that detects persons from a first image generated by a first camera for capturing a vehicle passing through a road; a first detection unit that detects the number of occupants in the vehicle captured in the first image; a second image analysis unit that detects persons from a second image generated by a second camera for capturing the vehicle passing through the road; a second detection unit that detects the number of occupants in the vehicle captured in the second image, a determination unit that determines whether or not the first image and the second image satisfy a predetermined condition; and a decision unit that decides, in a case where the first image and the second image satisfy the predetermined condition, the number of occupants in the vehicle with use of the number of occupants in the vehicle captured in the first image detected by the first detection unit and the number of occupants in the vehicle captured in the second image detected by the second detection unit.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Duy A Tran whose telephone number is (571)272-4887. The examiner can normally be reached Monday-Friday 8:00 am - 5:00 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL R MISTRY can be reached at (313)-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUY TRAN/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights