DETAILED ACTION
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on XXXXXXXXXXXXXX has been entered.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims X are canceled.
Claims X are amended.
Claims X are new.
Claims X are pending and have been examined.
This action is in reply to the papers filed on XXX (effective filing date xxx).
Claims 1-20 are pending and have been examined.
This action is in reply to the papers filed on 08/29/2025 (effective filing date 06/01/2023).
Information Disclosure Statement
The information disclosure statement(s) submitted: 06/01/2023, has/have been considered by the Examiner and made of record in the application file.
Amendment
The present Office Action is based upon the original patent application filed on 06/01/2023 as modified by the amendment filed on 08/29/2025.
Reasons For Allowance
Prior-Art Rejection withdrawn
Claims xxx are allowed. The closest prior art (See PTO-892, Notice of References Cited) does not teach the claimed:
The invention teaches… and the prior-art teaches…, however, the prior-art does not teach…
The closest prior-art (xxx) teach the features as disclosed in Non-final Rejection (xxxx), however, these cited references do not teach and the prior-art does not teach at least the following:
determining, at a second time after associating the information corresponding to the first loyalty card with the logged location, that a second user computing device is located within a specified distance of the logged location using a second positioning system of the second user computing device; in response to determining that the second user computing device is located within the specified distance of the logged location of the first user computing device at the first time of detecting: retrieving information corresponding to a second loyalty card, the second loyalty card being associated with the merchant and the second user computing device; and displaying, by the second user computing device, data describing the second loyalty card.
Claim Rejections - 35 USC §101 - Withdrawn
Per Applicant’s amendments and arguments and considering new guidance in the MPEP, the rejections are withdrawn. Specifically, in Applicant’s Remarks (dated 08/29/2025, pgs. 16-19), Applicant traverses the 35 USC §101 rejections arguing that the amended claims recite new limitations that are not abstract, amount to significantly more, are directed to a practical application, etc…
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1, 11, 17 are rejected under 35 U.S.C. 103 as being unpatentable over: Leonard et al. 2019/0283756; in view of Rezvan Behbahani et al. 2021/0339741.
18/327,312 – Claim 17. (Currently Amended) Leonard et al. 2019/0283756 teaches A method, comprising: retrieving map data associated with an environment, the map data indicates a mapped infrastructure element (Leonard et al. 2019/0283756 [0012 - the system may determine the driver is about to make a turn based on map data] As described in detail below, embodiments of the present disclosure assist a driver in performing a turn at an intersection. Embodiments of the present disclosure may first determine that the driver is attempting to make a turn at an intersection. The vehicle system may determine that the driver is about to make the turn based on actions such as, but not limited to, activation of a turn signal, a direction of the driver's gaze, or the position of a steering wheel. In one embodiment, the system may determine the driver is about to make a turn based on map data. In response to determining the driver is attempting to make a turn, the vehicle control system then determines a risk associated with the intersection, a perception associated with the driver, and a perception of the vehicle. The perception of the driver may be used to determine if the driver is distracted, and the risk is used to determine if is likely that the vehicle may contact another vehicle, be driven off the road, or another undesirable outcome. [0025 - object recognition logic may include any known or yet-to-be-developed object recognition algorithms that may be utilized to detect objects within an environment] As noted above, the logic stored on the one or more memory modules 106 may include object recognition logic. The object recognition logic may include any known or yet-to-be-developed object recognition algorithms that may be utilized to detect objects within an environment. Example object recognition algorithms include, but are not limited to, edge detection algorithms, corner detection algorithms, blob detection algorithms, and feature description algorithms (e.g., scale-invariant feature transform (“SIFT”), speeded up robust features (“SURF”), gradient location and orientation histogram (“GLOH”), and the like). The logic stored on the electronic control unit may also include speech recognition logic used to detect the words spoken by the driver and/or passengers within the vehicle 100. Any known or yet-to-be-developed speech recognition algorithms may be used for the speech recognition logic. [0038 - the perception of the vehicle is based on the information collected by various cameras and sensors of the object detection system 130 to view the surrounding environment (e.g., the intersection) of the vehicle] In addition to the driver's perception, the perception of the vehicle 100 may also be determined. The perception of the vehicle 100 may be determined based on objects located within or around the vicinity of the intersection 200 (FIG. 2) detected by the object detection system 130. More specifically, the perception of the vehicle is based on the information collected by various cameras and sensors of the object detection system 130 to view the surrounding environment (e.g., the intersection) of the vehicle 100. The perception of the vehicle 100 may also be determined based on one more operating conditions of the vehicle 100 detected by the vehicle operating condition sensor system 160.); collecting sensor data from a vehicle sensor system, the sensor data indicates a motorist perception of the mapped infrastructure element (Leonard et al. 2019/0283756 [0012 - the vehicle control system then determines a risk associated with the intersection, a perception associated with the driver, and a perception of the vehicle. The perception of the driver may be used to determine if the driver is distracted] As described in detail below, embodiments of the present disclosure assist a driver in performing a turn at an intersection. Embodiments of the present disclosure may first determine that the driver is attempting to make a turn at an intersection. The vehicle system may determine that the driver is about to make the turn based on actions such as, but not limited to, activation of a turn signal, a direction of the driver's gaze, or the position of a steering wheel. In one embodiment, the system may determine the driver is about to make a turn based on map data. In response to determining the driver is attempting to make a turn, the vehicle control system then determines a risk associated with the intersection, a perception associated with the driver, and a perception of the vehicle. The perception of the driver may be used to determine if the driver is distracted, and the risk is used to determine if is likely that the vehicle may contact another vehicle, be driven off the road, or another undesirable outcome. [0018 - the perception of the driver and the perception of the vehicle are based on a behavioral profile associated with the intersection and data gathered by a plurality of sensors of the vehicle] As explained in greater detail below, the perception of the driver and the perception of the vehicle are based on a behavioral profile associated with the intersection and data gathered by a plurality of sensors of the vehicle. The behavioral profile includes one or more characteristics that are representative of the intersection, and the sensors indicate conditions at the intersection such as visibility, cross-traffic, obstructions, driver attentiveness, and the like.); inferring, from the sensor data and the map data, a perception state of the mapped infrastructure element (Leonard et al. 2019/0283756 [0012] As described in detail below, embodiments of the present disclosure assist a driver in performing a turn at an intersection. Embodiments of the present disclosure may first determine that the driver is attempting to make a turn at an intersection. The vehicle system may determine that the driver is about to make the turn based on actions such as, but not limited to, activation of a turn signal, a direction of the driver's gaze, or the position of a steering wheel. In one embodiment, the system may determine the driver is about to make a turn based on map data. In response to determining the driver is attempting to make a turn, the vehicle control system then determines a risk associated with the intersection, a perception associated with the driver, and a perception of the vehicle. The perception of the driver may be used to determine if the driver is distracted, and the risk is used to determine if is likely that the vehicle may contact another vehicle, be driven off the road, or another undesirable outcome. [0018] As explained in greater detail below, the perception of the driver and the perception of the vehicle are based on a behavioral profile associated with the intersection and data gathered by a plurality of sensors of the vehicle. The behavioral profile includes one or more characteristics that are representative of the intersection, and the sensors indicate conditions at the intersection such as visibility, cross-traffic, obstructions, driver attentiveness, and the like. [0038] In addition to the driver's perception, the perception of the vehicle 100 may also be determined. The perception of the vehicle 100 may be determined based on objects located within or around the vicinity of the intersection 200 (FIG. 2) detected by the object detection system 130. More specifically, the perception of the vehicle is based on the information collected by various cameras and sensors of the object detection system 130 to view the surrounding environment (e.g., the intersection) of the vehicle 100. The perception of the vehicle 100 may also be determined based on one more operating conditions of the vehicle 100 detected by the vehicle operating condition sensor system 160.); and recommending a maintenance task to be performed based on the perception state of the mapped infrastructure element (Leonard et al. 2019/0283756 [0012 - the vehicle control system then determines a risk associated with the intersection, a perception associated with the driver, and a perception of the vehicle. The perception of the driver may be used to determine if the driver is distracted] As described in detail below, embodiments of the present disclosure assist a driver in performing a turn at an intersection. Embodiments of the present disclosure may first determine that the driver is attempting to make a turn at an intersection. The vehicle system may determine that the driver is about to make the turn based on actions such as, but not limited to, activation of a turn signal, a direction of the driver's gaze, or the position of a steering wheel. In one embodiment, the system may determine the driver is about to make a turn based on map data. In response to determining the driver is attempting to make a turn, the vehicle control system then determines a risk associated with the intersection, a perception associated with the driver, and a perception of the vehicle. The perception of the driver may be used to determine if the driver is distracted, and the risk is used to determine if is likely that the vehicle may contact another vehicle, be driven off the road, or another undesirable outcome. [0018 - the perception of the driver and the perception of the vehicle are based on a behavioral profile associated with the intersection and data gathered by a plurality of sensors of the vehicle] As explained in greater detail below, the perception of the driver and the perception of the vehicle are based on a behavioral profile associated with the intersection and data gathered by a plurality of sensors of the vehicle. The behavioral profile includes one or more characteristics that are representative of the intersection, and the sensors indicate conditions at the intersection such as visibility, cross-traffic, obstructions, driver attentiveness, and the like.).
Leonard et al. 2019/0283756 may not expressly disclose the “recommending a maintenance task to be performed” features, however Rezvan Behbahani et al. 2021/0339741 teaches (Rezvan Behbahani et al. 2021/0339741 [0093 - recommend system upgrades or maintenance for particular type of vehicles (e.g., sensors with improved accuracy, updated software for the perception system 820, the prediction system 822, the planning system 824, etc.) among other types of upgrades] The requirements system 852 may then select a set of requirements for the scenario/vehicle type (e.g., set of vehicles having similar equipment or capabilities) combination to send to the vehicles 802 as the requirement data 842. In some cases, the requirements system 852 may also recommend system upgrades or maintenance for particular type of vehicles (e.g., sensors with improved accuracy, updated software for the perception system 820, the prediction system 822, the planning system 824, etc.) among other types of upgrades.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Leonard et al. 2019/0283756 to include the features as taught by Rezvan Behbahani et al. 2021/0339741. One of ordinary skill in the art would have been motivated to do so to incorporate well-known features for recommending maintenance tasks which should prove to improve user experience, maximize profits, and optimize revenue.
18/327,312 – Claim 17. (Currently Amended) teaches A method, comprising:
retrieving map data associated with an environment, the map data indicates a mapped infrastructure element;
collecting sensor data from a vehicle sensor system, the sensor data indicates a motorist perception of the mapped infrastructure element ();
inferring, from the sensor data and the map data, a perception state of the mapped infrastructure element ();
and recommending a maintenance task to be performed based on the perception state of the mapped infrastructure element ().
18/327,312 – Claim 1. A system, comprising: a processor; and a memory storing machine-readable instructions that, when executed by the processor, cause the processor to: infer a perceived infrastructure element by a motorist within an environment based on sensor data collected from a sensor system of a vehicle; retrieve map data associated with the environment, wherein the map data indicates a mapped infrastructure element; and recommend a maintenance task to be performed based on the map data and the sensor data.
Claim 1, has similar limitations as of Claim(s) 17, therefore it is REJECTED under the same rationale as Claim(s) 17.
18/327,312 – Claim 11. A non-transitory machine-readable medium comprising instructions that, when executed by a processor, cause the processor to: collect sensor data from a sensor system of a vehicle; infer, from the sensor data, a perceived infrastructure element by a motorist within an environment; retrieve map data associated with the environment, wherein the map data indicates a mapped infrastructure element; and recommend, using a neural network model trained to identify maintenance tasks based on the map data and the sensor data, a maintenance task to be performed based on a comparison of the map data and the sensor data.
Claim 11, has similar limitations as of Claim(s) 17, therefore it is REJECTED under the same rationale as Claim(s) 17.
Claims 4 and 8 are rejected under 35 U.S.C. 103 as being unpatentable over: Leonard et al. 2019/0283756; in view of Rezvan Behbahani et al. 2021/0339741; in further view of Berkooz et al. 2023/0123958.
18/327,312 – Claim 4.(Currently Amended) Leonard et al. 2019/0283756 may not expressly teach the following, however, Berkooz et al. 2023/0123958 teaches The system of claim 1, wherein: the sensor data comprises vehicle operational data (Berkooz et al. 2023/0123958 [0012 - a dataset is also created in the controlled data collection step 20, for vehicle operational information from data collected from various sensors mounted in a vehicle during] More specifically, in one exemplary arrangement, the controlled data collection 20 is performed on different, known surfaces, at select and known speeds. For example, the data is collected on different road surfaces such as, for example, gravel and paved roads, but at varying speeds, such as low, medium, and high speeds. In one exemplary arrangement, the data collection may be performed on a test track, that has known road conditions. As the road surfaces are known, as are the speeds, the accuracy of the collected information can be verified to create a robust working reference data set. For creating the reference set, the data collection can be done both visually and manually, using known measurement techniques. In addition, a dataset is also created in the controlled data collection step 20, for vehicle operational information from data collected from various sensors mounted in a vehicle during a controlled drive on the known road surface. Sensors that may be employed include, for example, global positioning sensors, ball joint sensors, accelerometers, CAN data and wheel movement sensors. However, it is understood that other operational sensors may also be utilized.); and the machine-readable instruction to recommend the maintenance task (Berkooz et al. 2023/0123958 [0033 - the information may be utilized for predictive maintenance, such as road predictive maintenances] The operational processed data permits a determination of a condition of the road under the vehicle wheels and may provide a high-level overview of the status of roads and bridges from the aggregated vehicle data. Such information allows users to filter road class levels and allows detection of anomalies, such as potholes. It is contemplated in one exemplary arrangement, that a user may generate a summary report from the database, of a desired area or route that provides road surface changes and corresponding GPS locations, summary of potholes, including number and estimated sizes, as well as other identifiable road anomalies or even unidentified objects on the road. In some instances, the vehicle's onboard camera systems may be operated to take photos of anomalies under predetermined conditions and transmit those photos to the database to be accessed by a user. With knowledge of the road conditions, the operational parameters of the vehicle may be selectively manipulated to account for such road conditions, including improving the vehicle ride for the occupants, adjusting parameters to account for vehicle load, etc. In other exemplary arrangements, the information may be utilized for predictive maintenance, such as road predictive maintenances, or vehicle predictive maintenances.) further comprises a machine-readable instruction that, when executed by the processor, causes the processor to recommend the maintenance task based on the map data (Berkooz et al. 2023/0123958 [0037 - generate updates to a vehicle map system to designate rough road types, such as potholes or cobblestone roadway, to allow a driver to choose alternative routes to minimize wear on a vehicle] The system also includes a database into which the processed converted data may be stored. The database may be stored on the vehicle or on an external server. Information from the database may be used as inputs for other operational systems in the vehicle or external to the vehicle. For example, the database may be used to generate updates to a vehicle map system to designate rough road types, such as potholes or cobblestone roadway, to allow a driver to choose alternative routes to minimize wear on a vehicle.) and the vehicle operational data (Berkooz et al. 2023/0123958 [0012 - a dataset is also created in the controlled data collection step 20, for vehicle operational information from data collected from various sensors mounted in a vehicle during] More specifically, in one exemplary arrangement, the controlled data collection 20 is performed on different, known surfaces, at select and known speeds. For example, the data is collected on different road surfaces such as, for example, gravel and paved roads, but at varying speeds, such as low, medium, and high speeds. In one exemplary arrangement, the data collection may be performed on a test track, that has known road conditions. As the road surfaces are known, as are the speeds, the accuracy of the collected information can be verified to create a robust working reference data set. For creating the reference set, the data collection can be done both visually and manually, using known measurement techniques. In addition, a dataset is also created in the controlled data collection step 20, for vehicle operational information from data collected from various sensors mounted in a vehicle during a controlled drive on the known road surface. Sensors that may be employed include, for example, global positioning sensors, ball joint sensors, accelerometers, CAN data and wheel movement sensors. However, it is understood that other operational sensors may also be utilized.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Leonard et al. 2019/0283756 to include the features as taught by Berkooz et al. 2023/0123958. One of ordinary skill in the art would have been motivated to do so to incorporate well-known features for recommending maintenance tasks which should prove to improve user experience, maximize profits, and optimize revenue.
18/327,312 – Claim 8. (Currently Amended) Leonard et al. 2019/0283756 may not expressly teach the following, however, Berkooz et al. 2023/0123958 teaches The system of claim 1, wherein the machine-readable instruction to recommend the maintenance task further comprises an instruction that, when executed by the processor, causes the processor to recommend a repair of the mapped infrastructure element (Berkooz et al. 2023/0123958 [0033 - the information may be utilized for predictive maintenance, such as road predictive maintenances] The operational processed data permits a determination of a condition of the road under the vehicle wheels and may provide a high-level overview of the status of roads and bridges from the aggregated vehicle data. Such information allows users to filter road class levels and allows detection of anomalies, such as potholes. It is contemplated in one exemplary arrangement, that a user may generate a summary report from the database, of a desired area or route that provides road surface changes and corresponding GPS locations, summary of potholes, including number and estimated sizes, as well as other identifiable road anomalies or even unidentified objects on the road. In some instances, the vehicle's onboard camera systems may be operated to take photos of anomalies under predetermined conditions and transmit those photos to the database to be accessed by a user. With knowledge of the road conditions, the operational parameters of the vehicle may be selectively manipulated to account for such road conditions, including improving the vehicle ride for the occupants, adjusting parameters to account for vehicle load, etc. In other exemplary arrangements, the information may be utilized for predictive maintenance, such as road predictive maintenances, or vehicle predictive maintenances.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Leonard et al. 2019/0283756 to include the features as taught by Berkooz et al. 2023/0123958. One of ordinary skill in the art would have been motivated to do so to incorporate well-known features for recommending maintenance tasks which should prove to improve user experience, maximize profits, and optimize revenue.
Claims 2, 12, 18 are rejected under 35 U.S.C. 103 as being unpatentable over: Leonard et al. 2019/0283756; in view of Rezvan Behbahani et al. 2021/0339741; in further view of Berkooz et al. 2023/0123958; in further view of Barzelay et al. 2019/0188521.
18/327,312 – Claim 2. (Currently Amended) Leonard et al. 2019/0283756 may not expressly teach the following, however, Berkooz et al. 2023/0123958 teaches The system of claim 1, wherein: the sensor data comprises images of the environment (Berkooz et al. 2023/0123958 [0033 - vehicle's onboard camera systems may be operated to take photos of anomalies under predetermined conditions and transmit those photos to the database] The operational processed data permits a determination of a condition of the road under the vehicle wheels and may provide a high-level overview of the status of roads and bridges from the aggregated vehicle data. Such information allows users to filter road class levels and allows detection of anomalies, such as potholes. It is contemplated in one exemplary arrangement, that a user may generate a summary report from the database, of a desired area or route that provides road surface changes and corresponding GPS locations, summary of potholes, including number and estimated sizes, as well as other identifiable road anomalies or even unidentified objects on the road. In some instances, the vehicle's onboard camera systems may be operated to take photos of anomalies under predetermined conditions and transmit those photos to the database to be accessed by a user. With knowledge of the road conditions, the operational parameters of the vehicle may be selectively manipulated to account for such road conditions, including improving the vehicle ride for the occupants, adjusting parameters to account for vehicle load, etc. In other exemplary arrangements, the information may be utilized for predictive maintenance, such as road predictive maintenances, or vehicle predictive maintenances.); and the machine-readable instruction to recommend the maintenance task further comprises machine-readable instruction that, when executed by the processor, causes the processor to recommend the maintenance task based on a detected discrepancy between the map data and the images of the environment (Berkooz et al. 2023/0123958 [0033 - the information may be utilized for predictive maintenance, such as road predictive maintenances] The operational processed data permits a determination of a condition of the road under the vehicle wheels and may provide a high-level overview of the status of roads and bridges from the aggregated vehicle data. Such information allows users to filter road class levels and allows detection of anomalies, such as potholes. It is contemplated in one exemplary arrangement, that a user may generate a summary report from the database, of a desired area or route that provides road surface changes and corresponding GPS locations, summary of potholes, including number and estimated sizes, as well as other identifiable road anomalies or even unidentified objects on the road. In some instances, the vehicle's onboard camera systems may be operated to take photos of anomalies under predetermined conditions and transmit those photos to the database to be accessed by a user. With knowledge of the road conditions, the operational parameters of the vehicle may be selectively manipulated to account for such road conditions, including improving the vehicle ride for the occupants, adjusting parameters to account for vehicle load, etc. In other exemplary arrangements, the information may be utilized for predictive maintenance, such as road predictive maintenances, or vehicle predictive maintenances.).
Leonard et al. 2019/0283756 may not expressly disclose the “detected discrepancy” features, however, Barzelay et al. 2019/0188521 teaches (Barzelay et al. 2019/0188521 [0007 - comparing, by machine logic, the adjusted infrastructure object images with each other to determine a difference data set corresponding to a set of differences between at least two of the plurality of adjusted infrastructure object images; and (vii) analyzing, by machine logic, the difference data set to determine that a potential maintenance condition exists regarding the first infrastructure object] According to an aspect of the present invention, there is a method, computer program product and/or computer system for performing the following operations (not necessarily in the following order): (i) receiving a first video image that includes a first initial version infrastructure object image showing a first infrastructure object with the first initial version infrastructure object image being characterized by a first viewing vector; (ii) receiving a second video image that includes a second initial version infrastructure object image showing the first infrastructure object with the second initial version infrastructure object image being characterized by a second viewing vector that is at least approximately parallel to the first viewing vector; (iii) selecting the first initial version infrastructure image from the first video; (iv) analyzing, by machine logic, the second video image to determine that the second initial version infrastructure object image is a match with an identical instance of the first initial version infrastructure object image, with the analysis of the second video image including: (a) constructing, by machine logic, a three dimensional (3D) data model of at least a portion of the environment around the first infrastructure object based, at least in part, upon the first and second video images, and (b) determining the match based, at least in part, upon the respective relationships of the first and second initial infrastructure object images to the 3D data model; (v) adjusting, by machine logic, at least one of the first and second initial version infrastructure object image to obtain a plurality of adjusted infrastructure image objects respectively corresponding to the first and second initial version infrastructure object images, with the plurality of adjusted infrastructure object images showing the first infrastructure object aligned with itself across the plurality of adjusted infrastructure object images; (vi) comparing, by machine logic, the adjusted infrastructure object images with each other to determine a difference data set corresponding to a set of differences between at least two of the plurality of adjusted infrastructure object images; and (vii) analyzing, by machine logic, the difference data set to determine that a potential maintenance condition exists regarding the first infrastructure object.).
Before the effective filing date of the claimed invention, it would have been obvious for one of ordinary skill in the art to have modified Leonard et al. 2019/0283756 to include the features as taught by Barzelay et al. 2019/0188521. One of ordinary skill in the art would have been motivated to do so to incorporate well-known features for recommending roadway infrastructure maintenance tasks including neural network/machine learning tools which should prove to improve user experience, maximize profits, and optimize revenue.
18/327,312 – Claim 12. The non-transitory machine-readable medium of claim 11, wherein: the sensor data comprises images of the environment; and the instruction to recommend the maintenance task further comprises an instruction that, when executed by the processor, causes the processor to recommend the maintenance task based on a detected discrepancy between the map data and the images of the environment.
Claim 12, has similar limitations as of Claim(s) 2, therefore it is REJECTED under the same rationale as Claim(s) 2.
18/327,312 – Claim 18. The method of claim 17, wherein: the sensor data comprises images of the environment; and recommending the maintenance task further comprises recommending the maintenance task based on a detected discrepancy between the map data and the images of the environment.
Claim 18, has similar limitations as of Claim(s) 2, therefore it is REJECTED under the same rationale as Claim(s) 2.
No Prior-art Rejection / Potentially Allowable
Claims 3, 5-7, 9, 10, 13-16, 19, 20 cannot be rejected with prior-art. Individual claimed features are taught in the prior-art, however, the unique combination of features and elements are not taught by the prior-art without hindsight reasoning. These claims are further rejected to as being dependent upon a rejected base claim but might possibly be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
18/327,312 – Claim 3. The system of claim 2, wherein the instruction to recommend the maintenance task based on the detected discrepancy comprises an instruction that, when executed by the processor, causes the processor to identify that the mapped infrastructure element is not captured in an image of the environment of the vehicle.
18/327,312 – Claim 5. The system of claim 1, wherein the maintenance task is recommended based on: a similarity between the sensor data and historical sensor data collected from other vehicles passing through the environment; and a previously performed maintenance task associated with the historical sensor data.
18/327,312 – Claim 6. The system of claim 1, wherein: the machine-readable instructions further comprise an instruction that, when executed by the processor, causes the processor to retrieve an infrastructure record that indicates a status of the mapped infrastructure element; and the machine-readable instructions to recommend the maintenance task is further based on the infrastructure record.
18/327,312 – Claim 7. The system of claim 1, wherein: the machine-readable instructions further comprise an instruction that, when executed by the processor, causes the processor to retrieve an annotation regarding a historical maintenance task performed within the environment; and the machine-readable instructions to recommend the maintenance task is further based on the annotation.
18/327,312 – Claim 9. The system of claim 1, wherein the machine-readable instruction to recommend the maintenance task further comprises an instruction that, when executed by the processor, causes the processor to recommend an environmental repair, wherein the environmental repair increases a visibility of the mapped infrastructure element.
18/327,312 – Claim 10. The system of claim 1, wherein the machine-readable instruction to recommend the maintenance task further comprises an instruction that, when executed by the processor, causes the processor to recommend a non-road surface maintenance task.
18/327,312 – Claim 13. The non-transitory machine-readable medium of claim 11, wherein the machine-readable medium further comprises an instruction that, when executed by the processor, causes the processor to train the neural network model based on a completed maintenance task and the sensor data.
18/327,312 – Claim 14. The non-transitory machine-readable medium of claim 11, wherein: the machine-readable medium further comprises an instruction that, when executed by the processor, causes the processor to retrieve an infrastructure record that indicates a status of the mapped infrastructure element; and the instruction to recommend the maintenance task is further based on the infrastructure record.
18/327,312 – Claim 15. The non-transitory machine-readable medium of claim 11, wherein: the machine-readable medium further comprises an instruction that, when executed by the processor, causes the processor to retrieve an annotation regarding a historical maintenance task performed within the environment; and the instruction to recommend the maintenance task is further based on the annotation.
18/327,312 – Claim 16. The non-transitory machine-readable medium of claim 11, wherein the instruction to recommend the maintenance task comprises an instruction that, when executed by the processor, causes the processor to identify that the mapped infrastructure element indicated is not captured in an image of the environment of the vehicle.
18/327,312 – Claim 19. The method of claim 17: further comprising retrieving an infrastructure record that indicates a status of the mapped infrastructure element; and wherein recommending the maintenance task is further based on the infrastructure record.
18/327,312 – Claim 20. The method of claim 17: further comprising retrieving an annotation regarding a historical maintenance task performed within the environment; and wherein recommending the maintenance task is further based on the annotation.
Examiner’s Response to Arguments
Per Applicants’ amendments/arguments, the rejections are withdrawn.
Applicant's arguments have been considered but are moot in view of the new ground(s) of rejection.
Applicants’ amendments have necessitated the new grounds of rejection noted above.
Examiner’s Response: Claim Rejections – 35 USC §112
Per Applicants’ amendments/arguments, the rejections are withdrawn.
Applicant's arguments have been considered but are moot in view of the new ground(s) of rejection.
Applicants’ amendments have necessitated the new grounds of rejection noted above.
Examiner’s Response: Claim Rejections – 35 USC §101
Per Applicants’ amendments/arguments, the rejections are withdrawn. See notes above for additional reasoning and rationale for dropping 35 USC 101 rejection including Applicant’s amendments, arguments, lack of abstract idea, and practical integration.
Examiner’s Response: Claim Rejections – 35 USC § 103
Per Applicants’ amendments/arguments, the rejections are withdrawn. See notes above for additional reasoning and rationale for dropping prior-art rejection including Applicant’s amendments and arguments and unique combination of features and elements not taught by the prior-art without hindsight reasoning.
Applicant's arguments have been considered but are moot in view of the new ground(s) of rejection.
Applicants’ amendments have necessitated the new grounds of rejection noted above.
Any comments considered necessary by applicant must be submitted no later than the payment of the issue fee and, to avoid processing delays, should preferably accompany the issue fee. Such submissions should be clearly labeled “Comments on Statement of Reasons for Allowance.”
Conclusion
PERTINENT PRIOR ART – Patent Literature
The prior-art made of record and considered pertinent to applicant's disclosure.
Doutre et al. 2023/0177827 [0009 - infrastructure capable of being scanned by a camera mounted to a vehicle]
Kaku et al. 2021/0374432 [0044 - sensor calibration module 123 can be configured to receive data from the sensor system 120 and/or any other type of system capable of capturing information relating to the vehicle 100 and/or the external environment of the vehicle ]
Krehl et al. 2024/0312218 [0004 - computing system may then process the resultant hybrid BEV representation of the surrounding environment of the vehicle to, for example, derive a fused representation of the surrounding environment, derive various aspects of the road infrastructure on which the vehicle operates (e.g., lane markings, road topology, lane topology, crosswalks, etc.)]
Komori et al. 2024/0112149 [0009] (2): In the above-described aspect (1), the manager causes the provider to provide information for promoting maintenance of the road when a difference between a position of a road included in an image captured by the sensor device and a position of the road included in the reference image is greater than or equal to a threshold value.
Dickson et al. 2023/0306573 [0056 - neural network] [0024 - one or more machine learning techniques may be provided for identifying maintenance candidates. For instance, a machine learning model may be provided that analyzes sensor data collected from infrastructure assets, and outputs one or more candidates for a selected type of maintenance (e.g., reparative and/or preventive maintenance). Additionally, or alternatively, the machine learning model may output one or more images and/or portions of images that are identified as having a selected characteristic (e.g., exhibiting a selected type of damage), for example, to explain why the selected type of maintenance is recommended. Such a machine learning model may be trained using sensor data]
PERTINENT PRIOR ART – Non-Patent Literature (NPL)
The NPL prior-art made of record and considered pertinent to applicant's disclosure.
Toyota's high-tech city project set for launch: Woven City a 'test track' for smart homes, robotics, autonomous vehicles. In: ISE: Industrial & Systems Engineering at Work, Mar2025.
THIS ACTION IS MADE FINAL
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW T. SITTNER whose telephone number is (571) 270-7137 and email: matthew.sittner@uspto.gov. The examiner can normally be reached on Monday-Friday, 8:00am - 5:00pm (Mountain Time Zone).
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sarah M. Monfeldt can be reached on (571) 270-1833.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW T SITTNER/
Primary Examiner, Art Unit 3629b