DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Note
Regarding the phrase AI in the claims, examiner interpret AI as the artificial intelligent because it’s disclose in the paragraphs 10 and 12 of the specification.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 8, 11-14, 16 and 20 are rejected under 35 U.S.C. 102 (a)(2) as being anticipated by Aoude et al. US 20230186769.
Regarding claim 1, Aoude et al. disclose A road sentinel AI pylon comprising: a pylon having at least one sensor; (Aoude et al. US 20230186769 abstract; paragraphs [0006]-[0014]; [0023]-[0026]; [0051]; [0057]-[0060]; [0062]-[0069]; [0081]-[0089]; [0101]; [0110]; [0121]-[0127]; [0130]; [0133]; [0138]-[0141]; [0145]-[0151]; [0155]; [0158]; [0176]; [0179]; [0182]; [0189]- [0190]; [0209]; [0210]; figures 1-18;)
Implementations may include one or a combination of two or more of the following features. The equipment includes a roadside equipment. There is a housing for the equipment and the sensor is attached to the housing. The warning is sent by broadcasting the warning for receipt by any of the ground transportation entities at or near the intersection that can receive the warning. The machine learning model includes an artificial intelligence model. The intersection includes a non-signalized intersection. The intersection includes a signalized intersection. The transportation network includes a road network. The ground transportation entities include vulnerable road users. The ground transportation entities include vehicles. The imminent dangerous situation includes a collision. The ground transportation entity the device of which cannot receive the warning from the wireless communication device includes a vehicle. The ground transportation entity the device of which can receive the warning from the wireless communication device includes a pedestrian crossing a road at a crosswalk. There is another communication device to communicate with a central server. The device of one of the ground transportation entity includes a mobile communication device (Aoude et al. par. 9). The system (e.g., the RSE or SRSE associated with the sensors) collects streams of data from all sensors. When the system is first put into operation, to help with equipment calibration and functionality, an initial rule-based model may be deployed. In the meantime, sensor data (e.g., speed and distance from radar units, images and video from cameras) is collected and stored locally at the RSE in preparation, in some implementations, to be transferred to a remote computer that is powerful enough to build an AI model of the behavior of the different entities of the intersection using this collected data. In some cases, the RSE is a SRSE capable of generating the AI model itself (Aoude et al. par. 155). FIG. 16 depicts a pedestrian 8102 crossing a crosswalk 8103. The crosswalk 8103 can be at an intersection or a mid-block crosswalk across a stretch of road between intersections. A camera 8101 is used to monitor the sidewalk 8104. The global locations of the boundaries of the field of view 8105 of the camera 8101 can be determined at the time of installation. The field of view 8105 is covered by a predetermined number of pixels that is reflected by the specifications of camera 8101. A road entity 8102 can be detected within the field of view of the camera and its global location can be calculated. The speed and heading of the road entity 8102 can also be determined from its displacement at s times. The path of the road entity 8102 can be represented by breadcrumbs 8106 which is a train of locations that the entity 8102 has traversed. This data can be used to build a virtual PSM message. The PSM message can then be broadcast to all entities near the intersection (Aoude et al. par. 210).
According to the cited passages and figures, examiner interpret a camera 8101 as a sensor mount on a post as show in the figure 13 and examiner interpret a post as a pylon.
said at least one sensor is configured to locate a fixed object and locate and track a moving or changing object;
Fixed location: Sensors situated at the intersection can be adjusted and fixed to sense in a specific direction that can be optimal for detecting important targets. This will help the processing software to better detect objects. As an example, if a camera has a fixed view, the background (non-moving objects and structures) information in the fixed view can be easily detected and used to improve the identification and classification of relatively important moving entities (Aoude et al. par. 126). A connected entity 1001, is traveling along a path 1007. The entity 1001 has a green light 1010. A non-connected entity 1002 is traveling along a path 1006. It has a red light 1009 but will be making a right on red along path 1006. This will place it directly in the path of the entity 1001. A dangerous situation is imminent since the entity 1001 is unaware of the entity 1002. Because the entity 1002 is a non-connected entity it is unable to broadcast (e.g., advertise) its position and heading to other entities sharing the intersection. Moreover, the entity 1001, even though it is connected, is unable to “see” the entity 1002 which is obscured by the building 1008. There is a risk of the entity 1001 going straight through the intersection and hitting the entity 1002 (Aoude et al. par. 130). Camera data, by contrast, can represent an image of the field of view at any moment in time. Lidar data may provide the locations of points in 3D space that correspond to the points of reflection of the laser beam emitted from the lidar at a specific time and heading (Aoude et al. par. 138). o get a unified view (representation) of the intersection, fusion of data from different types of sensors is useful. For purposes of fusion, the data from various sensor is translated into a common (unified) format that is independent of the sensor used. The data included in the unified format from all of the sensors will include the global location, speed, and heading of every entity using the intersection independently of how it was detected (Aoude et al. par. 139). FIG. 16 depicts a pedestrian 8102 crossing a crosswalk 8103. The crosswalk 8103 can be at an intersection or a mid-block crosswalk across a stretch of road between intersections. A camera 8101 is used to monitor the sidewalk 8104. The global locations of the boundaries of the field of view 8105 of the camera 8101 can be determined at the time of installation. The field of view 8105 is covered by a predetermined number of pixels that is reflected by the specifications of camera 8101. A road entity 8102 can be detected within the field of view of the camera and its global location can be calculated. The speed and heading of the road entity 8102 can also be determined from its displacement at s times. The path of the road entity 8102 can be represented by breadcrumbs 8106 which is a train of locations that the entity 8102 has traversed. This data can be used to build a virtual PSM message. The PSM message can then be broadcast to all entities near the intersection (Aoude et al. par. 210).
According to the cited passages and figures, examiner interpret sidewalk 8104, traffic light and building as the fix objects. For example building 1008 show in the figure 8 and traffic light show in the figure 7 as the fix objects that can capture by the sensors. Paragraph 139 disclose all the sensors include the global location so the sensors can locate a fixed object.
said pylon further including a wireless communication system and said pylon further includes a notification that provides information to a driver or an authority.
The RSE also includes or can make use of communication equipment 20 to communicate by wire or wireless with other RSEs, and with OBEs, OPEs, local or central servers, and other data processing units. The RSE can use any available standard for communication with other equipment. The RSE may use wired or wireless Internet connections for downloading and uploading data to other equipment, the cellular network to send and receive messages from other cellular devices, and a dedicated radio device to communicate to infrastructure devices and other RSEs at the intersection or other location (Aoude et al. par. 65). When the machine learning (AI) model is completed at the server, it is downloaded to the RSE through the Internet, for example. The RSE then applies current data captured from the sensors to the AI model to cause it to predict intent and behavior, to determine when a dangerous situation is imminent, and to trigger corresponding alerts that are distributed (e.g., broadcast) to the vehicles and other ground transportation entities and to the vulnerable road users and drivers as early warnings in time to enable the vulnerable road users and drivers to undertake collision avoidance steps (Aoude et al. par. 158).
Regarding claim 2, Aoude et al. disclose The road sentinel AI pylon according to claim 1, wherein said notification is a text message, an email, a changeable sign, an LED light or a light.
On Person Equipment (OPE) 46 which can be, but is not limited to, a mobile phone, wearable device, or any other device that is capable of being worn by, held by, attached to, or otherwise interfacing with a person or animal. OPEs can include or be coupled to data processing units 48, data storage 50, and communication equipment 52 if needed. In some implementations, an OPE serves as a dedicated communication unit for a non-vehicular vulnerable road user. In some cases, the OPE can also be used for other purposes. The OPE may have a component to provide visual, audio, or haptic alerts to the vulnerable road user (Aoude et al. par. 68).
Regarding claim 3, Aoude et al. disclose The road sentinel AI pylon according to claim 1, wherein sensor is selected from the group consisting of a camera, an infrared detector, thermal imaging, a night vision camera, a LiDAR, a microphone, a radar, and a sonar.
Sensors 201 and sensor controllers 207 that may include, but are not limited to, external cameras, lidars, radars, ultrasonic sensors or any device that may be used to detect nearby objects or people or other ground transportation entities. Sensors 201 may also include additional kinematic sensors, global positioning receivers, and internal and local microphones and cameras (Aoude et al. par. 92). Good vantage point: Infrastructure poles, beams, and support cables usually have an elevated vantage point. The elevated vantage points allow for a more general view of the intersection. This is like an observation tower at an airport where controllers have a full view of most of the important and vulnerable users on the ground. For ground transportation entities, by contrast, the views from the vantage point of sensors (camera, lidar, radar, etc. . . . or others) can be obstructed or disrupted by a truck in a neighboring lane, direct sunlight, or other interference. The sensors at the intersection can be chosen to be immune or less susceptible to such interference. A radar, for example, is not affected by sunlight and will remain effective during the evening commute. A thermal camera will be more likely to detect a pedestrian in a bright light situation where the view of an optical camera becomes hindered (Aoude et al. par. 125).
Regarding claim 8, Aoude et al. disclose The road sentinel AI pylon according to claim 1, wherein said at least one sensor tracks a group consisting of pedestrians, animals, vehicles, traffic cones, construction zones, and debris.
There is a housing for the equipment and the sensor is attached to the housing. The warning is sent by broadcasting the warning for receipt by any of the ground transportation entities at or near the intersection. The machine learning model includes an artificial intelligence model. The training data and the motion data include at least one of speed, location, or heading. The training data and motion data may also include intent, posture, direction of look, or interaction with other vulnerable road users, such as in a group (Aoude et al par. 7). The motion data generated by the sensors located in the vicinity of the crosswalk is segmented based on corresponding zones in the vicinity of the crosswalk. The electronic sensors are used to generate motion related data representing physical properties of the vulnerable road user. Trajectory information about the vulnerable road user is derived from motion data generated by the sensor (Aoude et al. par. 13). Vulnerable road user can include pedestrians, cyclists, road workers, people on wheelchairs, scooters, self-balancing devices, battery powered personal transporters, animal driven carriages, guide or police animals, farm animals, herds, and pets (Aoude et al. par. 69).
Regarding claim 11, Aoude et al. disclose The road sentinel AI pylon according to claim 1, wherein said object learning model is communicated over said wireless communication system to a core data center.
Implementations may include one or a combination of two or more of the following features. The equipment includes a roadside equipment. The vulnerable roadway user includes a pedestrian, animal, or cyclist. The device associated with the vulnerable roadway user includes a smart watch or other wearable, a smart phone, or another mobile device. The other ground transportation entity includes a motorized vehicle. The device associated with the other ground transportation entity includes a smart phone or another mobile device. The machine learning model is provided to the equipment located in the vicinity of the crosswalk by a remote server through the Internet. The machine learning model is generated at the equipment located in the vicinity of the crosswalk. The machine learning model is trained using motion data generated by the sensors located in the vicinity of the crosswalk. Motion data generated by the sensors located in the vicinity of the crosswalk is sent to a server for use in training the machine learning model. The motion data generated by the sensors located in the vicinity of the crosswalk is segmented based on corresponding zones in the vicinity of the crosswalk. The electronic sensors are used to generate motion related data representing physical properties of the vulnerable road user. Trajectory information about the vulnerable road user is derived from motion data generated by the sensor (Aoude et al. par. 13). In general, in an aspect, equipment is located at a level crossing of a transportation network that includes an intersection of a road, a pedestrian crossing, and a rail line. The equipment includes inputs to receive data from sensors oriented to monitor road vehicles and pedestrians at or near the level crossing and to receive phase and timing data for signals on the road and on the rail line. A wireless communication device is included to send to a device of one of the ground transportation entities, pedestrians, or rail vehicles on the rail line, a warning about a dangerous situation at or near the level crossing (Aoude et al. par. 24). A processing unit 105 that will acquire and use the data generated from the sensors as well as incoming data from the communication units 103, 104. The processing unit will process and store the data locally and, in some implementations, transmit the data for remote storage and further processing. The processing unit will also generate messages and alerts that are broadcast or otherwise sent through wireless communication facilities to nearby pedestrians, motor vehicles, or other ground transportation entities, and in some cases to signs or other infrastructure presentation devices. The processing unit will also periodically report the health and status of all the RSE systems to a remote server for monitoring (Aoude et al. par. 86).
According to the cited passages and figures, examiner interpret a remote server as a core data center and the wireless communication device is for wireless communication. For example the alerts that are broadcast or otherwise sent through wireless communication facilities.
Regarding claim 12, Aoude et al. disclose The road sentinel AI pylon according to claim 5, wherein said 3-D environment is stored on a local object tracking database.
As discussed above, different types of sensors can be used to detect different types of entities. The information from these sensors can be different, e.g., inconsistent with respect to the location or motion parameters that its data represents or the native format of the data or both. For example, radar data typically includes speed, distance, and maybe additional information such as the number of moving and stationary entities that are in the field of view of the radar. Camera data, by contrast, can represent an image of the field of view at any moment in time. Lidar data may provide the locations of points in 3D space that correspond to the points of reflection of the laser beam emitted from the lidar at a specific time and heading. In general, each sensor provides data in a native format that closely represents the physical quantities they measure (Aoude et al. par. 138). The system (e.g., the RSE or SRSE associated with the sensors) collects streams of data from all sensors. When the system is first put into operation, to help with equipment calibration and functionality, an initial rule-based model may be deployed. In the meantime, sensor data (e.g., speed and distance from radar units, images and video from cameras) is collected and stored locally at the RSE in preparation, in some implementations, to be transferred to a remote computer that is powerful enough to build an AI model of the behavior of the different entities of the intersection using this collected data. In some cases, the RSE is a SRSE capable of generating the AI model itself (Aoude et al. par. 155).
Regarding claim 13, Aoude et al. disclose The road sentinel AI pylon according to claim 12, wherein said 3-D environment is communicated to a mobile client.
As discussed above, different types of sensors can be used to detect different types of entities. The information from these sensors can be different, e.g., inconsistent with respect to the location or motion parameters that its data represents or the native format of the data or both. For example, radar data typically includes speed, distance, and maybe additional information such as the number of moving and stationary entities that are in the field of view of the radar. Camera data, by contrast, can represent an image of the field of view at any moment in time. Lidar data may provide the locations of points in 3D space that correspond to the points of reflection of the laser beam emitted from the lidar at a specific time and heading. In general, each sensor provides data in a native format that closely represents the physical quantities they measure (Aoude et al. par. 138). VPSM (virtual basic safety message) messages enable the implementation of pedestrian to vehicle (P2V), pedestrian to infrastructure (P2I), pedestrian to devices (P2D), vehicle to pedestrian (V2P), infrastructure to pedestrians (I2P), and devices to pedestrians (D2P) applications that would have been otherwise difficult to implement. (Aoude et al. par. 209).
Regarding claim 14, Aoude et al. disclose The road sentinel AI pylon according to claim 13, wherein said mobile client receives real-time information regarding said fixed object and said moving and said changing object around said road sentinel AI pylon.
The system can be tailored to make predictions for that particular intersection and to send alerts to the entities in the vicinity of the device broadcasting the alerts. For this purpose, the system will use sensors to derive data about the dangerous entity and pass the current readings from the sensors through the trained model. The output of the model then can predict a dangerous situation and broadcast a corresponding alert. The alert, received by connected entities in the vicinity, contains information about the dangerous entity so that the receiving entity can analyze that information to assess the threat posed to it by the dangerous entity. If there is a threat, the receiving entity can either take action itself (e.g., slowing down) or notify the driver of the receiving entity using a human machine interface based on visual, audio, haptic, or any kind of sensory stimulation. An autonomous entity may take action itself to avoid a dangerous situation (Aoude et al. par. 58). The alert can also be sent directly through the cellular or other network to a mobile phone or other device equipped to receive alerts and possessed by a pedestrian. The system identifies potential dangerous entities at the intersection and broadcasts (or directly sends) alerts to a pedestrian's personal device having a communication unit. The alert may, for example, prevent a pedestrian from entering a crosswalk and thus avoid a potential accident (Aoude et al par. 59).
Regarding claim 16, Aoude et al. disclose The road sentinel AI pylon according to claim 14, wherein said moving and said changing object around said road sentinel AI pylon is an individual, an animal or a contraflow driving vehicle(s).
There is a housing for the equipment and the sensor is attached to the housing. The warning is sent by broadcasting the warning for receipt by any of the ground transportation entities at or near the intersection. The machine learning model includes an artificial intelligence model. The training data and the motion data include at least one of speed, location, or heading. The training data and motion data may also include intent, posture, direction of look, or interaction with other vulnerable road users, such as in a group (Aoude et al par. 7). The motion data generated by the sensors located in the vicinity of the crosswalk is segmented based on corresponding zones in the vicinity of the crosswalk. The electronic sensors are used to generate motion related data representing physical properties of the vulnerable road user. Trajectory information about the vulnerable road user is derived from motion data generated by the sensor (Aoude et al. par. 13). Vulnerable road user can include pedestrians, cyclists, road workers, people on wheelchairs, scooters, self-balancing devices, battery powered personal transporters, animal driven carriages, guide or police animals, farm animals, herds, and pets (Aoude et al. par. 69).
Regarding claim 20, Aoude et al. disclose The road sentinel AI pylon according to claim 1, further includes at least a second road sentinel AI pylon that communicates over said wireless communication system to said road sentinel AI pylon.
The RSE also includes or can make use of communication equipment 20 to communicate by wire or wireless with other RSEs, and with OBEs, OPEs, local or central servers, and other data processing units. The RSE can use any available standard for communication with other equipment. The RSE may use wired or wireless Internet connections for downloading and uploading data to other equipment, the cellular network to send and receive messages from other cellular devices, and a dedicated radio device to communicate to infrastructure devices and other RSEs at the intersection or other location (Aoude et al. par. 65).
According to the cited passages and figures, examiner interpret other RSEs as a second road sentinel AI pylon.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Aoude et al. US 20230186769 in view of Ogihara et al. US 20220067198.
Regarding claim 4, Aoude et al. teach all the limitation in the claim 4.
Aoude et al. do not explicitly teach The road sentinel AI pylon according to claim 1, wherein sensor maps movement of an object as an anonymous data block.
Ogihara et al. teach The road sentinel AI pylon according to claim 1, wherein sensor maps movement of an object as an anonymous data block. (Ogihara et al. US 20220067198 abstract; paragraphs [0005]-[0008]; [0017]; [0085]; [0089]-[0094]; [0109]-[0110]; [0116]-0119]; figures 1-5;)
Embodiments of the present disclosure relate to a computer-implemented method, an associated computer system and computer program product for dynamically controlling a surrounding environment based on anonymized data collected from devices positioned within the surrounding environment. The computer-implemented method comprising: instructing, by a processor, a collection of data comprising biological data, location data, individual IDs and time data from a plurality of client devices; identifying, by the processor, as a function of the location data collected, map data and historical entry/exit data, a surrounding physical environment for each of the plurality of client devices corresponding to a time the location data is collected; converting, by the processor, the collection of data into anonymous data at an anonymization level established by a set of collection conditions at the time of the collection of the data; storing, by the processor, the anonymous data in a database; periodically extracting, by the processor, a portion of the anonymous data associated with the surrounding physical environment from the database; analyzing, by the processor, the portion of the anonymous data associated with the surrounding physical environment; and as a function of analyzing the portion of anonymous data associated with the surrounding physical environment, applying, by the processor, a modification to an environmental control of the surrounding physical environment (Ogihara et al. par. 5). The data collection gateway 212 receiving the collected data from one or more client devices 201 may further manage and process the collected data to identify the physical environment 202 each client device 201 is positioned within at the time of data collection based on the location data and individual IDs provided by the client device 201. Embodiments of the data collection gateway 212 may deploy the environmental retrieval module 214 to identify the physical environment, as shown via line 265. Based on the location data, the environmental module 214 may query a mapping data repository 229 for mapping data that further pinpoints and/or describes in greater detail the physical environment 202 associated with the location data, as shown by line 269. Moreover, based on the individual IDs collected from the client devices 201, embodiments of the environmental retrieval module 214 may retrieve entry/exit data from a repository comprising entry/exit history 227 as shown in FIG. 2b. The entry/exit history 227 may describe client device 201 movements as the client device 201 and/or user ingresses and/or egresses to and from the physical environment. The querying of the entry/exit history and retrieval of the entry/exit data is depicted by line 267 (Ogihara et al. par. 109).
Therefore, it would have been obviously to one of ordinary skill in the art before the effective filing date of the claim invention to modify the sensors monitor system of Aoude et al. reference by incorporating the data collection and anonymization techniques as taught by Ogihara et al. reference. The combination of Ogihara et al. reference into Aoude et al. reference for enhance privacy protection.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Aoude et al. US 20230186769 in view of Rawlings US 5034812.
Regarding claim 5, Aoude et al. teach The road sentinel AI pylon according to claim 1, that creates a 3-D rendered environment around said pylon.
Implementations may include one or a combination of two or more of the following features. The sensors include at least two of: radar, lidar, and a camera. The data received from one of the sensors includes image data of a field of view at successive moments. The data received from one of the sensors includes points of reflection in 3D space. The data received from one of the sensors includes distance from the sensor and speed. The global unified representation represents locations of the ground transportation entities in a common reference frame. Two sensors from which the data is received are mounted in fixed positions at or near the intersection and have at least partially non-overlapping fields of view. One of the sensors includes radar and the converting of the data includes determining locations of ground transportation entities from a known location of the radar and distances from the radar to the ground transportation entities. One of the sensors includes a camera and the converting of the data includes determining locations of ground transportation entities from a known location, direction of view, and tilt of the camera and the locations of the ground transportation entities within an image frame of the camera (Aoude et al. par. 23). As discussed above, different types of sensors can be used to detect different types of entities. The information from these sensors can be different, e.g., inconsistent with respect to the location or motion parameters that its data represents or the native format of the data or both. For example, radar data typically includes speed, distance, and maybe additional information such as the number of moving and stationary entities that are in the field of view of the radar. Camera data, by contrast, can represent an image of the field of view at any moment in time. Lidar data may provide the locations of points in 3D space that correspond to the points of reflection of the laser beam emitted from the lidar at a specific time and heading. In general, each sensor provides data in a native format that closely represents the physical quantities they measure (Aoude et al. par. 138).
According to the cited passages and figures, examiner interpret points of reflection in 3D space mention in paragraphs 23 and 138 as the 3D rendered environment around the intersection.
Aoude et al. do not explicitly teach wherein said pylon is pre-loaded with topographical baseline data.
Rawlings teach wherein said pylon is pre-loaded with topographical baseline data. (Rawlings US 5034812 abstract; col. 1 lines 34-63; col. 2 lines 56-66; col. 5 lines 23-44; figures 1-3;)
The detector 52 provides outputs in respect of identified objects to a trigonometric range and size processor 61, an iconometric range and size processor 62 and a kinematic range processor 63. An output is also provided to a subtended image extractor 64 which calculates the angle subtended at the camera 1 by the object viewed and supplies this information to the processors 61 and 62. The processors 61 and 62 also receive inputs from a topographical map 70, a pre-loaded intelligence map 71 and a new object and update map 72 which are associated with respective ones of the object data stores 73 to 75. The topographical map 70 contains information about the topography, that is, ground contours and permanent features on the ground on which is superimposed the information about the location of other objects which may be more transient. The intelligence map 71 contains additional information about the location of objects within the topographical map which may be gathered and loaded just prior to the flight to bring up to date the map information. The new object and update map 72 and its associated data store 75 is supplied with information from external sources (not shown) such as data links, crew inputs or the new object characterizer 54. The crew input could, for example, include a helmet-mounted sight and a speech recognizer so that the sight could be aimed at a target which is then vocally named by the pilot. Information about the appearance of the named object would then be read out of the store 75 for use in subsequent target tracking (Rawlings col. 5 lines 23-44).
Therefore, it would have been obviously to one of ordinary skill in the art before the effective filing date of the claim invention to modify the sensors monitor system of Aoude et al. reference by incorporating a topographical map and a pre-loaded intelligence map that contain ground contours and permanent features on the ground on which is superimposed the information about the location of other objects which may be more transient and the intelligence map contains additional information about the location of the objects within the topographical map as taught by Rawlings reference. The combination of Rawlings reference into Aoude et al. reference would enhance the accuracy of monitoring the objects in the environment.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Aoude et al. US 20230186769 in view of Rawlings US 5034812 and further in view of Johnson US 20100069035.
Regarding claim 6, the combination of Aoude et al. and Rawlings teach all the limitation in the claim 5.
The combination of Aoude et al. and Rawlings do not explicitly teach The road sentinel AI pylon according to claim 5, wherein said 3-D environment is created as a decimal degree data.
Johnson teaches The road sentinel AI pylon according to claim 5, wherein said 3-D environment is created as a decimal degree data. (Johnson US 20100069035 abstract; paragraph [0937]; [1257]; figures 7-11;)
PointSet provides means for defining a set of points for a variety of applications. Points of a PointSet may describe a single point (i.e. one point record), a line segment, a polygon, a point with radius, a two dimensional area, a three dimensional area in space, or any other multi-dimensional region. An optional dimension qualifier (i.e. 2D or 3D; default=2D) specifies whether or not the set of points are for two dimensional space or three dimensional space. Alternate embodiments support higher dimensions for certain applications, for example to describe another universe dimension as straightforward as time, or a situational location (e.g. extending a point record definition), or as complex as a string theory dimension. If point records can be specified for the dimension qualifier(s), any dimension(s) may be used. An optional point type qualifier (i.e. Geo, Cartesian or Polar; default=Geo) specifies the type of points in the set wherein each point is a record of appropriate data. Alternate embodiments support other type qualifiers for certain applications, for example to describe lines, arcs, or regions containing an infinite set of points (e.g. extending a point record definition for describing a collection of points), or to specify different models (e.g. Geodetic, Polar Cylindrical, Polar Spherical, etc). When a "text string" format is used for the PointSet, it is preferably null terminated (e.g. null included in ANSI encoded length) and an appropriate syntax is used to identify point record components (e.g. comma), and to delimit point records (e.g. semicolon) in the set of points (e.g. "+33.27,-97.4;+34.1,-97.3;+34.13,-97.12;" specifies a two dimensional Geo polygon PointSet (i.e. point records of latitude, longitude decimal degree pairs) and "3D/Geo; +33.27,-97.4,4500F;+34.1,-97.3,1L;+34.13,-97.21,2000Y;+34.3,-97.1,2000Y;+- 34.89,-97.08,2000Y" specifies a three dimensional Geo polygon solid region in space PointSet (i.e. point records of latitude, longitude, altitude decimal degree tuples)) (Johnson par. 937).
Therefore, it would have been obviously to one of ordinary skill in the art before the effective filing date of the claim invention to substitute point of reflection in 3D space in Aoude et al. and Rawlings reference with a pointset for 3D like latitude, longitude and decimal degree pairs as taught by Johnson reference because both of them are provide the location data in the 3D environment.
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Aoude et al. US 20230186769 in view of Rawlings US 5034812 and further in view of Kim et al. US 20190324148.
Regarding claim 7, the combination of Aoude et al. and Rawlings teach The road sentinel AI pylon according to claim 5, wherein said 3-D environment includes at least two of landscape, signage, sidewalks, lanes, lane markings, center lane, sub lanes,
As show in the figures 6, 8, 16 and 18 of Aoude et al. reference the 3D environment include a road, vehicle, pedestrian, traffic light, street line, lanes, lane marking and multiple lanes
The combination of Aoude et al. and Rawlings do not explicitly teach pot holes, speed bumps, and dips.
Kim et al. teach pot holes, speed bumps, and dips. (Kim et al. US 20190324148 abstract paragraph [0003]; figures 1-6;)
LiDAR sensors can be particularly useful for mapping the environment (e.g., by generating a three-dimensional point cloud) because laser range measurements are particularly accurate and can be generated with reasonable speed required for autonomous or partial autonomous operations. Conventionally, LiDAR data can be processed using height thresholding, where objects can be classified as objects/hazards based on their height relative to the vehicle ground. However, even with a well calibrated setup (with a known geometric relationship between LiDAR sensor coordinates and vehicle coordinates, and which can be eventually extended further into a real-world coordinate system), conventional techniques such as height thresholding cannot reliably differentiate between ground/free space and objects/hazards due to real-world environmental conditions including the fluctuation of LiDAR coordinate system cause by dynamic motion of the vehicle during navigation and non-flat driving surfaces roads (varying slopes, pots holes, dips, speed bumps, etc.). Improved processing to distinguish between free space and objects/hazards is required (Kim et al. par. 3).
Therefore, it would have been obviously to one of ordinary skill in the art before the effective filing date of the claim invention to substitute the data received from radar, lidar and a camera in Aoude et al. and Rawlings reference with the data like potholes, dips and speed bump as taught by Johnson reference because both of the data received by sensor like Lidar in the 3D environment.
Claim 9 is rejected under 35 U.S.C. 103 as being unpatentable over Aoude et al. US 20230186769 in view of Vallespi-Gonzalez et al. US 20190171912.
Regarding claim 9, Aoude et al. teach all the limitation in the claim 8.
However, Aoude et al. do teach machine learning module but Aoude et al. do not explicitly teach The road sentinel AI pylon according to claim 8, wherein said pylon uses data from said at least one sensor to create an object learning model.
Vallespi-Gonzalez et al. teach The road sentinel AI pylon according to claim 8, wherein said pylon uses data from said at least one sensor to create an object learning model. (Vallespi-Gonzalez et al. US 20190171912 abstract; paragraphs [0020]-[0022]; [0045]; [0028]; [0089]; [0108]; [0112]; [0119]; [0123]; figures 1-9)
At 606, the method 600 can include determining, characteristics of the one or more portions of sensor data. In some embodiments, the method 600 can include determining characteristics of the one or more portions of sensor data (e.g., the one or more portions of sensor data in 602 and/or 604) in a second stage of the multiple stage classification, one or more second stage characteristics of the one or more portions of sensor data based in part on a second machine-learned model. For example, the second stage computing system 230 can determine one or more characteristics of one or more portions of sensor data received from the one or more sensor devices 202 and/or the first stage computing system 210, and can perform the determination using a machine-learned object detection and recognition model that has been trained to detect and/or recognize one or more objects including streets, buildings, the sky, vehicles, pedestrians, and/or cyclists (Vallespi-Gonzalez et al. par. 123).
Therefore, it would have been obviously to one of ordinary skill in the art before the effective filing date of the claim invention to substitute the machine learning model in Aoude et al. reference with a machine-learned object detection and recognition model as taught by Vallespi-Gonzalez et al. reference, as both of the machine learning models provide a similar functionality, as learning from data collected by sensors to perform object recognition and detection.
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Aoude et al. US 20230186769 in view of Vallespi-Gonzalez et al. US 20190171912 and further in view of Janzen et al. US 20180096595.
Regarding claim 10, the combination of Aoude et al. and Vallespi-Gonzalez et al. teach all the limitation in the claim 9.
The combination of Aoude et al. and Vallespi-Gonzalez et al. do not explicitly teach The road sentinel AI pylon according to claim 9, wherein said object learning model is used to create a historical model of changes in location of said pedestrians, animals, vehicles, traffic cones, construction zones, and debris.
Janzen et al. teach The road sentinel AI pylon according to claim 9, wherein said object learning model is used to create a historical model of changes in location of said pedestrians, animals, vehicles, traffic cones, construction zones, and debris. (Janzen et al. US 20180096595 abstract; paragraphs [0028]-[0030];[0048]; [0051]-[0057]; [0071]-[0079]; [0088]-[0089]; [0130]; [0154]-[0155]; [0165]-[0168]; [0176]-[0181]; figures 1-14;)
Turning now to the drawings, traffic signal control systems and methods in accordance with various embodiments of the invention are illustrated. Traffic signal control systems in accordance with many embodiments of the invention incorporate a plurality of traffic optimization systems located at a network of intersections. Each traffic optimization system can include at least one camera mounted at, near, or on one or more traffic signal poles located at an intersection. One or more sensor processing units within the traffic optimization system can process images captured by the one or more cameras to detect any of a variety of objects including (but not limited to) automobiles (with varying degrees of specificity), busses, cyclists, pedestrians, emergency vehicles, boats, and/or trolleys/light rail trains. In a number of embodiments, the traffic optimization system also detects historical motion of a detected object and/or attempts to predict future motion of the detected object. In this way, the traffic optimization system can determine movement of detected objects such as (but not limited to) vehicles and/or pedestrians with respect to an intersection and can control the traffic signals accordingly. For example, the traffic optimization system can send a message to a traffic signal controller to delay traffic signal phasing based upon the presence of pedestrians within the crosswalk and/or the absence of cars stopped waiting at the intersection. As can readily be appreciated, the specific manner in which the traffic optimization system can communicate with a traffic controller to adjust phasing of traffic signals at an intersection is largely dependent upon the requirements of a given application (Janzen et al. par. 48). There are also several features which can be extracted from the surroundings which can help improve traffic safety. These features are highlighted in FIG. 11. A background model of the surroundings can be automatically generated in many motion detection algorithms. Using this background model, features such as road wear, locations of potholes, fading lane markings, and/or accumulating debris or obstructions in the roadway can be identified. The background image also gives insight into the current road visibility conditions such as sun glare and/or fog/smog, and provides a method for determining whether the road is wet, snowy, and/or icy (Janzen et al. par. 176).
Therefore, it would have been obviously to one of ordinary skill in the art before the effective filing date of the claim invention to combine Aoude et al. and Vallespi-Gonzalez et al. with Janzen et al. by comprising the teaching of Janzen et al. into the system of Aoude et al. and Vallespi-Gonzalez et al.. The motivation to combine these arts is to provide a historical motion of a detected object and attempts to predict future motion from Janzen et al. reference into Aoude et al. and Vallespi-Gonzalez et al. reference so it can help road user to avoid collision.
Claims 15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Aoude et al. US 20230186769 in view of Rawlings US 5034812 and further in view of Cherewka US 20170061791.
Regarding claim 15, the combination of Aoude et al. and Rawlings teach all the limitation in the claim 5.
The combination of Aoude et al. and Rawlings do not explicitly teach The road sentinel AI pylon according to claim 5, wherein said 3-D environment is communicated over a Mesh network protocol.
Cherewka teaches The road sentinel AI pylon according to claim 5, wherein said 3-D environment is communicated over a Mesh network protocol. (Cherewka US 20170061791 abstract; paragraphs [0005]-[0007]; [0024]-[0032]; [0045]-[0053]; figures 1-9;)
Various embodiments of the present invention provide a fully automated and intelligent monitoring system designed to replace human workers (e.g., flaggers) in highway and road construction work zones. This invention utilizes computer vision (IP cameras) and artificial intelligence and establishes a wireless mesh network through various on-site traffic control devices. The system maps out critical areas of the work zone, manages and optimizes traffic in real time, communicates with drivers, and assists contractors with features that include live video monitoring and event recording, all using a minimum number of physical devices. This invention completely replaces flaggers in almost any situation, allows supervisors to monitor multiple projects at once, and provides construction firms valuable video recording and analytics of their projects. As a vision-based intelligent platform, various alternate embodiments of this invention include integrated telematics, traffic analysis, remote diagnostics, site mapping, and fleet management, and connects vehicles to heavy equipment using vehicle-to-infrastructure (V2I) communications. The system and devices of the present invention create an intelligent work zone that is similar to smart infrastructure/smart roads initiatives, but that exists within a construction zone. Managing traffic and replacing/eliminating flaggers in works zones is an important aspect of this invention; however, the present invention is essentially a robotic system for monitoring and managing work zones, with embedded artificial intelligence for controlling traffic. The system may be employed for any kind of roadside work involving temporary lane closures, including paving, water mains and gas lines for utilities, tree trimming for power lines, cable laying for Internet providers, etc. Further application for the traffic control system may be traffic control for mining, security checkpoints such as entrance gates for military bases, or for government or private facilities, highway toll booths, railroad crossings, pedestrian crossings, etc. (Cherewka par. 45).
Therefore, it would have been obviously to one of ordinary skill in the art before the effective filing date of the claim invention to substitute the network in Aoude et al. and Rawlings reference with a mesh network as taught by Cherewka reference because both of the network provide a similar function for communication between multiple devices.
Regarding claim 1