Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
Claim Objections
Claims 8-14 objected to because of the following informalities:
It is unclear if claims 8-14 are intended to be examined in dependent or independent form. The claims are currently written in independent claim form while maintaining dependence to limitations within claim 1.
Due to the format confusion and fees worksheet filed, the office will examine the claims in dependent form. The office recommends rewriting the claims in proper dependent claim form or to structure the claims in proper independent claim form.
Claims 10-14 are objected to for containing limitations directed to a different statutory category (“moving image map processing device”, “moving image map processing system”, and “vehicle having moving image map processing capability”) from the independent claim (“Update image map method”)
The office recommends rewriting the claims in proper independent claim form to maintain the desired statutory category. Additionally, if the claims are rewritten as independent, their perspective dependent claims must additionally be reapplied separately with appropriate dependence.
Claim 9 is objected to due to confusing language. More specifically, the text of claim 9 claims to execute the steps before the limitations of claim 1 (“comprising the following steps before executing the update image map method of claim 1”), but contains the steps that make up the limitations of claim 1 (“executing the step of receiving the vehicle information; executing the step of processing the images; executing the step of writing the processed images to the corresponding location on the image map; and executing the update image map method.”).
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3 and 5-14 are rejected under 35 U.S.C. 103 as being unpatentable over IKEN (WO2023012096A1) in view of BOURKE (US 20200334928 A1).
Regarding claim 1:
IKEN discloses:
An update image map method applied to a vehicle, comprising: (see at least IKEN, ¶ 0055, "In particular, the (raw) sensor data and/or the environmental data are used by means of (at least one and/or more) sensor device(s) to detect a vehicle's environment during a journey and/or during the operation of the vehicle (the user) and especially generated and/or determined in real time. Preferably, the map data is also generated while the vehicle is driving. ")
updating an image map (see at least IKEN, ¶ 0054, "Preferably, the backend can be an area of interest (English: area of interest). Specify the Region of Interest (ROI) of the (captured and/or to be captured) vehicle environment. Preferably, a signal received from a backend (with regard to the specification of an area of interest) triggers the vehicle to generate recordings in that area of interest and /or to obtain map data relating to that area of interest. "; ¶ 0055)
receiving a vehicle information and capturing images from one or more cameras; (see at least IKEN, ¶ 0010, "The method comprises generating a multitude of images of spatially resolved environmental data (such as images from a camera) using at least one sensor device (such as a camera), wherein the ground surface of the vehicle environment is located at least partially within the vehicle environment detected by the at least one sensor device, wherein the images differ from each other, in particular in pairs, by at least one recording parameter, in particular a recording location and/or a recording direction. ")
processing the images based on lens characteristics of the one or more cameras and (see at least IKEN, ¶ 0028, "Preferably, for each sensor device for detecting the vehicle environment, which is used in particular for generating the high-resolution map or determining the map data, at least one recording parameter and/or several recording parameters, in particular a characteristic value for a (relative) recording location (within the vehicle) and/or for a recording direction and/or for a recording angle and/or a field of view (FOV) and/or calibration data of the sensor device, are stored on a storage device of the vehicle and can be retrieved in particular by the map generation device."; ¶ 0055)
the vehicle information; and (see at least IKEN, ¶ 0035, "In a further preferred method, the map generation device determines and/or retrieves odometry data of the vehicle or data derived therefrom to determine the respective vehicle position. This offers the advantage that, for example, even in vehicle positions where other localization methods are not possible because no GPS signal can be received or is available, a (precise) localization of the vehicle and determination of the vehicle position can still be carried out. "; ¶ 0036, "By adding odometry, the recordings (such as camera images) from different vehicle positions are combined. Thus, an image or map is formed along the (vehicle) trajectory."; ¶ 0037 "Preferably, the odometry data is determined and/or retrieved based on data from the chassis sensors. For example, odometry data can be based on quantities (such as control variables) from the vehicle's steering and/or acceleration sensor systems. to be determined. Preferably, the odometer data are characteristic of a (momentary) steering angle and/or a (momentary) acceleration of the vehicle and/or of a distance traveled (with respect to a reference point) and/or of a revolution of the vehicle wheels. ")
writing the processed images to a corresponding location on the image map based on the received vehicle information. (see at least IKEN, ¶ 0011, "Furthermore, for at least one, and preferably for each, recording of the plurality of recordings, at least one recording size is determined by a, in particular processor-based, map generation device of the vehicle, wherein the recording size is characteristic for at least one recording parameter of the respective recording (and/or the respective sensor device that generates the recording). Furthermore, the map generation device determines map data by combining the multitude of recordings depending on the, in particular, at least one determined recording size. In particular, when combining the multitude of recordings, at least one determined recording parameter and preferably the at least one determined recording parameter are taken into account."; ¶ 0016, "In other words, a swarm map is created from accumulated recordings (such as images). This map or composite image (a composite picture) contains significantly more information than the individual images separately. Thus, the accumulated or composite recording represents a larger (ground) area, but also advantageously enables further subsequent steps, such as deriving features from the map data and/or correcting the map data. In particular, this enables Combining multiple images (such as several map images) to create a ground plane, so that map information can be derived from the map (generated thereby). Furthermore, the proposed creation of this map using a swarm is comparatively less complex. "; ¶ 0075, "Fig. 1 shows a schematic representation of a vehicle 1 with a map generation device 10 according to one embodiment according to the invention. The vehicle 1 has at least one sensor device 12 for detecting the vehicle's surroundings, in this case a front camera. The recordings of spatially resolved environmental data captured by this sensor device 12, which are camera images captured here, are transmitted to the map generation device 10 of the vehicle (in particular via a wired communication link) and preferably processed by the map generation device 10. ")
EXAMINERS NOTE: While IKEN does not explicitly discuss the processing of images based on lens characteristics, IKEN discloses the collection of data regarding the "field of view" and "calibration data" which would be recognized by a person having ordinary skill in the art to encompass the characteristics of the lens of a camera.
IKEN does not disclose, but BOURKE teaches:
receiving a message used to determine whether conditions for (see at least BOURKE, ¶ 0063, “According to another aspect of the present invention there is provided an apparatus for analysing collision data, the apparatus comprising: means (for example in the form of a suitably programmed processor and associated memory) and for receiving telematics data relating to a collision; means for (for example in the form of a suitably programmed processor and associated memory) determining a collision severity based on said data; and means for (for example in the form of a suitably programmed processor and associated memory) notifying an entity of said collision based on the determined severity of the collision.”)
determining whether the conditions for (see at least BOURKE, ¶ 0063; ¶ 0132, “Due to the unpredictable nature of collisions, it is necessary to record data during the entirety of a journey to ensure that data corresponding to a collision is captured. However due to the privacy concerns of users it is desirable to record, and/or to store, only necessary data. As such, only recorded data related to a time period preceding and following a collision is retained: for example data from the beginning of the journey during which a collision occurs, or from an amount of time preceding and following the collision. In an example a data file from a 6-10 second excerpt of a collision is stored and analysed. Data not related to a collision is overwritten or deleted. This recording/deletion may occur in a circular fashion so that only the last ‘X’ minutes of data is ever available.”)
when it is determined that the conditions for (see at least BOURKE, ¶ 0063; ¶ 0132)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, with a reasonable expectation of success, the stitching of multiple cameras for mapping an environment with remote triggering of IKEN to include the accident detection where in certain components crossing a threshold trigger a notification within BOURKE to effectively yield an emergency recording system that triggers mapping and recording on emergency events.
EXAMINERS NOTE: It should be noted the BOURQUE does not teach the triggering of a map update (hence references to "updating an image map" being crossed out for BOURKE references), however a person having ordinary skill in the art would recognize that the triggering of an event to record is inherently different from the recording act itself. The claimed invention does not improve upon the communication of messages to trigger. IKEN discloses the limitation of "updating an image map".
Regarding claim 2:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
the vehicle information includes a current position and a current heading angle of the vehicle. (see at least IKEN, ¶ 0035, "In a further preferred method, the map generation device determines and/or retrieves odometry data of the vehicle or data derived therefrom to determine the respective vehicle position. This offers the advantage that, for example, even in vehicle positions where other localization methods are not possible because no GPS signal can be received or is available, a (precise) localization of the vehicle and determination of the vehicle position can still be carried out. "; ¶ 0036, "By adding odometry, the recordings (such as camera images) from different vehicle positions are combined. Thus, an image or map is formed along the (vehicle) trajectory."; ¶ 0037, "Preferably, the odometry data is determined and/or retrieved based on data from the chassis sensors. For example, odometry data can be based on quantities (such as control variables) from the vehicle's steering and/or acceleration sensor systems. to be determined. Preferably, the odometer data are characteristic of a (momentary) steering angle and/or a (momentary) acceleration of the vehicle and/or of a distance traveled (with respect to a reference point) and/or of a revolution of the vehicle wheels. ")
Regarding claim 3:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
the vehicle information is obtained from one or any combination of the following: a vehicle information system; and a navigation and positioning system. (see at least IKEN, ¶ 0035, "In a further preferred method, the map generation device determines and/or retrieves odometry data of the vehicle or data derived therefrom to determine the respective vehicle position. This offers the advantage that, for example, even in vehicle positions where other localization methods are not possible because no GPS signal can be received or is available, a (precise) localization of the vehicle and determination of the vehicle position can still be carried out. "; ¶ 0036, "By adding odometry, the recordings (such as camera images) from different vehicle positions are combined. Thus, an image or map is formed along the (vehicle) trajectory."; ¶ 0037, "Preferably, the odometry data is determined and/or retrieved based on data from the chassis sensors. For example, odometry data can be based on quantities (such as control variables) from the vehicle's steering and/or acceleration sensor systems. to be determined. Preferably, the odometer data are characteristic of a (momentary) steering angle and/or a (momentary) acceleration of the vehicle and/or of a distance traveled (with respect to a reference point) and/or of a revolution of the vehicle wheels. ")
Regarding claim 5:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
updating the image map (see at least IKEN, ¶ 0054, "Preferably, the backend can be an area of interest (English: area of interest). Specify the Region of Interest (ROI) of the (captured and/or to be captured) vehicle environment. Preferably, a signal received from a backend (with regard to the specification of an area of interest) triggers the vehicle to generate recordings in that area of interest and /or to obtain map data relating to that area of interest. "; ¶ 0055, "In particular, the (raw) sensor data and/or the environmental data are used by means of (at least one and/or more) sensor device(s) to detect a vehicle's environment during a journey and/or during the operation of the vehicle (the user) and especially generated and/or determined in real time. Preferably, the map data is also generated while the vehicle is driving. ")
a relative displacement of the vehicle exceeds a threshold; (see at least IKEN, ¶ 0031, "In a preferred method, the different recording location of at least two recordings is given by a different vehicle position relative to the vehicle's surroundings. In particular, the location where at least two images and preferably a multitude of images are taken results (at least also) from a respective different vehicle position of the vehicle (at the time the respective images are taken). The vehicle can move during the recording process (along a vehicle trajectory), in particular continuously and/or without interruption. Preferably, the recordings are generated at such a temporal and spatial interval (with regard to the vehicle environment) that (essentially) each recording has an overlap area with another recording and/or each recording depicts and/or represents at least partially the same area of the recorded vehicle environment as at least one other recording. "; ¶ 0051, "Preferably, the generated high-resolution map and/or map data is transmitted from the vehicle to an external storage device. The map and/or map data can be transmitted to the external storage device as soon as a predetermined storage size or a predetermined portion of the maximum storage size on the vehicle's storage device, in particular the ring buffer, is reached. It is also conceivable that the vehicle transmits the high-resolution map and/or map data to the external storage device at regular time and/or distance intervals. This offers the advantage that the usually limited storage space on the vehicle's internal storage device is not exceeded. Preferably, only the (fully created or generated) map (possibly with its associated, especially global, position data) is transmitted to the external storage device, but not the multitude of recordings. In particular, it is not preferred that the large number of recordings be transmitted to an external storage device. This offers the advantage that the volume of data to be transmitted is limited and/or reduced, since the (vehicle-internal or vehicle-specific) correction values and/or recording parameters do not need to be transmitted (to the backend). ")
a time elapsed since the previous step of writing the processed images exceeds a time threshold. (see at least IKEN, ¶ 0031; ¶ 0051)
EXAMINERS NOTE: Even though IKEN does not explicitly disclose the tracking of image collection by crossing a threshold of time, IKEN does anticipate the collection and arrangement of images in a temporal and spatial interval. Using broadest reasonable interpretation would recognize that the collection intervals of images requires the use of a timer or displacement tracking to keep the intervals consistent.
IKEN does not disclose, but BOURKE teaches:
the conditions for (see at least BOURKE, ¶ 0063, “According to another aspect of the present invention there is provided an apparatus for analysing collision data, the apparatus comprising: means (for example in the form of a suitably programmed processor and associated memory) and for receiving telematics data relating to a collision; means for (for example in the form of a suitably programmed processor and associated memory) determining a collision severity based on said data; and means for (for example in the form of a suitably programmed processor and associated memory) notifying an entity of said collision based on the determined severity of the collision.”)
a relative heading angle of the vehicle exceeds another threshold; (see at least BOURKE, ¶ 0050, “Determining a collision severity and/or determining events associated with the collision may comprise modelling or simulating a collision to evaluate a collision severity and/or events associated with the collision. Determining other features may comprise modelling or simulating a collision to evaluate said features. Such features may include an impact area, a cost of repair, a likelihood of a component being damaged, a probability of injury, a magnitude or direction of a collision, a rotation of a vehicle, an injury compensation cost, a measure of liability, a momentum of a further entity, a velocity of a further entity, a time of collision, a time of highest magnitude of acceleration, a force caused by a collision, an elasticity of an object involved in a collision, a spring constant, a momentum change of an object involved in a collision, an event preceding a collision, an acceleration above a threshold level, a change in direction and/or speed, and/or a loss of traction.”)
a current speed of the vehicle exceeds a speed threshold; (see at least BOURKE, ¶ 0050)
a steering angle of a steering mechanism of the vehicle exceeds a steering angle threshold; and (see at least BOURKE, ¶ 0193, “In some embodiments, said telematics data is used to detect skids or loss of traction events. This may involve methods for detecting differences in the vehicle heading (as defined by the forwards-backwards axis) and the direction of travel (as defined by the velocity/momentum). This may involve information from onboard diagnostics information, such as ABS signals, and steering, accelerator, and brake inputs.”)
EXAMINERS NOTE: It should be noted the BOURQUE does not teach the triggering of a map update (hence references to "updating an image map" being crossed out for BOURKE references), however a person having ordinary skill in the art would recognize that the triggering of an event to record is inherently different from the recording act itself. The claimed invention does not improve upon the communication of messages to trigger.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, with a reasonable expectation of success, the stitching of multiple cameras for mapping an environment with remote triggering of IKEN to include the accident detection where in certain components crossing a threshold trigger a notification within BOURKE to effectively yield an emergency recording system that triggers mapping and recording on emergency events.
Regarding claim 6:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
updating the image map. (see at least IKEN, ¶ 0011, "Furthermore, for at least one, and preferably for each, recording of the plurality of recordings, at least one recording size is determined by a, in particular processor-based, map generation device of the vehicle, wherein the recording size is characteristic for at least one recording parameter of the respective recording (and/or the respective sensor device that generates the recording). Furthermore, the map generation device determines map data by combining the multitude of recordings depending on the, in particular, at least one determined recording size. In particular, when combining the multitude of recordings, at least one determined recording parameter and preferably the at least one determined recording parameter are taken into account."; ¶ 0016, "In other words, a swarm map is created from accumulated recordings (such as images). This map or composite image (a composite picture) contains significantly more information than the individual images separately. Thus, the accumulated or composite recording represents a larger (ground) area, but also advantageously enables further subsequent steps, such as deriving features from the map data and/or correcting the map data. In particular, this enables Combining multiple images (such as several map images) to create a ground plane, so that map information can be derived from the map (generated thereby). Furthermore, the proposed creation of this map using a swarm is comparatively less complex. "; ¶ 0075, "Fig. 1 shows a schematic representation of a vehicle 1 with a map generation device 10 according to one embodiment according to the invention. The vehicle 1 has at least one sensor device 12 for detecting the vehicle's surroundings, in this case a front camera. The recordings of spatially resolved environmental data captured by this sensor device 12, which are camera images captured here, are transmitted to the map generation device 10 of the vehicle (in particular via a wired communication link) and preferably processed by the map generation device 10. ")
IKEN does not disclose, but BOURKE teaches:
setting the conditions for (see at least BOURKE, ¶ 0063, “According to another aspect of the present invention there is provided an apparatus for analysing collision data, the apparatus comprising: means (for example in the form of a suitably programmed processor and associated memory) and for receiving telematics data relating to a collision; means for (for example in the form of a suitably programmed processor and associated memory) determining a collision severity based on said data; and means for (for example in the form of a suitably programmed processor and associated memory) notifying an entity of said collision based on the determined severity of the collision.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, with a reasonable expectation of success, the stitching of multiple cameras for mapping an environment with remote triggering of IKEN to include the accident detection where in certain components crossing a threshold trigger a notification within BOURKE to effectively yield an emergency recording system that triggers mapping and recording on emergency events.
Regarding claim 7:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
organizing the image map; and (see at least IKEN, ¶ 0006, "A system for correcting HD map images is known from DE 102019 133 536 A1. For this purpose, the system is designed to capture ground view image data of a map anomaly hotspot detected by the vehicles from a large number of vehicles. The ground-level image data is converted into bird's-eye view image data using homography, outliers are removed from the bird's-eye view image data, and an error reduction mechanism is applied. A sequence of consecutive images from the image data after error reduction are combined to create a high-resolution image. This is compared to a high-resolution image of the hotspot to determine changes to the high-resolution image. From US patent 2018/0188041 A1, a high-resolution mapping system is known which receives sensor data from a vehicle traveling along a route and uses the data to generate a high-resolution map for use in vehicles. A machine learning-based result filter detects poor fitting results and prevents them from being fed into a subsequent global pose optimization. ")
pausing for a period of time. (see at least IKEN, ¶ 0031, "In a preferred method, the different recording location of at least two recordings is given by a different vehicle position relative to the vehicle's surroundings. In particular, the location where at least two images and preferably a multitude of images are taken results (at least also) from a respective different vehicle position of the vehicle (at the time the respective images are taken). The vehicle can move during the recording process (along a vehicle trajectory), in particular continuously and/or without interruption. Preferably, the recordings are generated at such a temporal and spatial interval (with regard to the vehicle environment) that (essentially) each recording has an overlap area with another recording and/or each recording depicts and/or represents at least partially the same area of the recorded vehicle environment as at least one other recording. "; ¶ 0051, "Preferably, the generated high-resolution map and/or map data is transmitted from the vehicle to an external storage device. The map and/or map data can be transmitted to the external storage device as soon as a predetermined storage size or a predetermined portion of the maximum storage size on the vehicle's storage device, in particular the ring buffer, is reached. It is also conceivable that the vehicle transmits the high-resolution map and/or map data to the external storage device at regular time and/or distance intervals. This offers the advantage that the usually limited storage space on the vehicle's internal storage device is not exceeded. Preferably, only the (fully created or generated) map (possibly with its associated, especially global, position data) is transmitted to the external storage device, but not the multitude of recordings. In particular, it is not preferred that the large number of recordings be transmitted to an external storage device. This offers the advantage that the volume of data to be transmitted is limited and/or reduced, since the (vehicle-internal or vehicle-specific) correction values and/or recording parameters do not need to be transmitted (to the backend). ")
EXAMINERS NOTE: Even though IKEN does not explicitly disclose the tracking of image collection by crossing a threshold of time, IKEN does anticipate the collection and arrangement of images in a temporal and spatial interval. Using broadest reasonable interpretation would recognize that the collection intervals of images requires the use of a timer or displacement tracking to keep the intervals consistent.
IKEN does not disclose, but BOURKE teaches:
when it is determined that the conditions for updating the image map are not met, executing one or a combination of the following steps: (see at least BOURKE, ¶ 0063, “According to another aspect of the present invention there is provided an apparatus for analysing collision data, the apparatus comprising: means (for example in the form of a suitably programmed processor and associated memory) and for receiving telematics data relating to a collision; means for (for example in the form of a suitably programmed processor and associated memory) determining a collision severity based on said data; and means for (for example in the form of a suitably programmed processor and associated memory) notifying an entity of said collision based on the determined severity of the collision.”; ¶ 0132, “Due to the unpredictable nature of collisions, it is necessary to record data during the entirety of a journey to ensure that data corresponding to a collision is captured. However due to the privacy concerns of users it is desirable to record, and/or to store, only necessary data. As such, only recorded data related to a time period preceding and following a collision is retained: for example data from the beginning of the journey during which a collision occurs, or from an amount of time preceding and following the collision. In an example a data file from a 6-10 second excerpt of a collision is stored and analysed. Data not related to a collision is overwritten or deleted. This recording/deletion may occur in a circular fashion so that only the last ‘X’ minutes of data is ever available.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine, with a reasonable expectation of success, the stitching of multiple cameras for mapping an environment with remote triggering of IKEN to include the accident detection where in certain components crossing a threshold trigger a notification within BOURKE to effectively yield an emergency recording system that triggers mapping and recording on emergency events.
Regarding claim 8:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
reading a corresponding part of the image map based on the vehicle information; and transmitting the corresponding part of the read image map to a display module and displaying the corresponding part of the read image map on the display module. (see at least IKEN, ¶ 0076, "For this purpose, the map generation unit 10 creates a map composed of the recordings from the sensor unit or, in this case, from the camera images. For this purpose, the map generation unit 10 projects the camera images onto the ground or the ground surface of the vehicle's surroundings, creating a so-called "BirdView" view of the vehicle or a map display in bird's-eye view. "; ¶ 0077, "Reference symbol 30 indicates an (optical) output device by means of which such a generated map or information derived therefrom can be output to a user. ")
Regarding claim 9:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
comprising the following steps before executing the update image map method of claim 1:
creating a blank image map; (see at least IKEN, ¶ 0020, "Preferably, the multitude of recordings is transmitted, in particular via a wired communication link (in the vehicle) or characteristic data to the, in particular processor-based, map generation device and/or stored on a (vehicle-internal) storage device, preferably a ring buffer. ")
executing the step of receiving the vehicle information; (see at least IKEN, ¶ 0010, "The method comprises generating a multitude of images of spatially resolved environmental data (such as images from a camera) using at least one sensor device (such as a camera), wherein the ground surface of the vehicle environment is located at least partially within the vehicle environment detected by the at least one sensor device, wherein the images differ from each other, in particular in pairs, by at least one recording parameter, in particular a recording location and/or a recording direction. ")
executing the step of processing the images; (see at least IKEN, ¶ 0011, "Furthermore, for at least one, and preferably for each, recording of the plurality of recordings, at least one recording size is determined by a, in particular processor-based, map generation device of the vehicle, wherein the recording size is characteristic for at least one recording parameter of the respective recording (and/or the respective sensor device that generates the recording). Furthermore, the map generation device determines map data by combining the multitude of recordings depending on the, in particular, at least one determined recording size. In particular, when combining the multitude of recordings, at least one determined recording parameter and preferably the at least one determined recording parameter are taken into account.")
executing the step of writing the processed images to the corresponding location on the image map; and (see at least IKEN, ¶ 0013, "In particular, a high-resolution map is obtained by combining the multitude of images, especially depending on the respective at least one determined image size. This high-resolution map preferably depicts an area of the surroundings or the vehicle environment, and in particular a ground surface of the vehicle environment (around the vehicle). ")
executing the update image map method. (see at least IKEN, ¶ 0075, "Fig. 1 shows a schematic representation of a vehicle 1 with a map generation device 10 according to one embodiment according to the invention. The vehicle 1 has at least one sensor device 12 for detecting the vehicle's surroundings, in this case a front camera. The recordings of spatially resolved environmental data captured by this sensor device 12, which are camera images captured here, are transmitted to the map generation device 10 of the vehicle (in particular via a wired communication link) and preferably processed by the map generation device 10. ")
EXAMINERS NOTE: While IKEN does not explicitly disclose the creation of a blank image map, a person having ordinary skill in the art would know that when starting a map with no data, the file created is empty (blank). Additionally, regardless of step, it is well known in the art that data cannot be collected and stored without the initial creation of a file to store data within.
EXAMINERS NOTE: A person having ordinary skill in the art would recognize the steps recited between creating the blank image map and updating the image map can be done in any order to yield the same outcome.
Regarding claim 10:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
at least one processor, used to execute a plurality of instructions to implement the update image map method of claim 1; and (see at least IKEN, ¶ 0069, "The present invention further relates to a computer program or computer program product, comprising program means, in particular a program code, which represents or encodes at least one and preferably several process steps (individually or in combination with each other) of the method according to the invention and preferably one of the described preferred embodiments and is designed for execution by a processor device. ")
a memory, used to store the image map. (see at least IKEN, ¶ 0020, "Preferably, the multitude of recordings is transmitted, in particular via a wired communication link (in the vehicle) or characteristic data to the, in particular processor-based, map generation device and/or stored on a (vehicle-internal) storage device, preferably a ring buffer. ")
Regarding claim 11:
IKEN in view of BOURKE discloses the limitation within claim 8 and IKEN further discloses:
at least one processor, used to execute a plurality of instructions to implement the display image map method of claim 8; (see at least IKEN, ¶ 0069, "The present invention further relates to a computer program or computer program product, comprising program means, in particular a program code, which represents or encodes at least one and preferably several process steps (individually or in combination with each other) of the method according to the invention and preferably one of the described preferred embodiments and is designed for execution by a processor device. ")
a memory, used to store the image map; and (see at least IKEN, ¶ 0020, "Preferably, the multitude of recordings is transmitted, in particular via a wired communication link (in the vehicle) or characteristic data to the, in particular processor-based, map generation device and/or stored on a (vehicle-internal) storage device, preferably a ring buffer. ")
an input and output interface, used to transmit the corresponding part of the read image map to the display module. (see at least IKEN, ¶ 0077, "Reference symbol 30 indicates an (optical) output device by means of which such a generated map or information derived therefrom can be output to a user.")
Regarding claim 12:
IKEN in view of BOURKE discloses the limitation within claim 11 and IKEN further discloses:
the moving image map processing device and the display module of claim 11. (see at least IKEN, ¶ 0076, "For this purpose, the map generation unit 10 creates a map composed of the recordings from the sensor unit or, in this case, from the camera images. For this purpose, the map generation unit 10 projects the camera images onto the ground or the ground surface of the vehicle's surroundings, creating a so-called "BirdView" view of the vehicle or a map display in bird's-eye view. "; ¶ 0077, "Reference symbol 30 indicates an (optical) output device by means of which such a generated map or information derived therefrom can be output to a user.")
Regarding claim 13:
IKEN in view of BOURKE discloses the limitation within claim 12 and IKEN further discloses:
the moving image map processing system and the one or more cameras of claim 12. (see at least IKEN, ¶ 0010, "The method comprises generating a multitude of images of spatially resolved environmental data (such as images from a camera) using at least one sensor device (such as a camera), wherein the ground surface of the vehicle environment is located at least partially within the vehicle environment detected by the at least one sensor device, wherein the images differ from each other, in particular in pairs, by at least one recording parameter, in particular a recording location and/or a recording direction. "; ¶ 0076, "For this purpose, the map generation unit 10 creates a map composed of the recordings from the sensor unit or, in this case, from the camera images. For this purpose, the map generation unit 10 projects the camera images onto the ground or the ground surface of the vehicle's surroundings, creating a so-called "BirdView" view of the vehicle or a map display in bird's-eye view. ")
Regarding claim 14:
IKEN in view of BOURKE discloses the limitation within claim 9 and IKEN further discloses:
at least one processor, used to execute a plurality of instructions to implement the initialization image map method of claim 9; and a memory, used to store the image map. (see at least IKEN, ¶ 0020, "Preferably, the multitude of recordings is transmitted, in particular via a wired communication link (in the vehicle) or characteristic data to the, in particular processor-based, map generation device and/or stored on a (vehicle-internal) storage device, preferably a ring buffer. ")
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over IKEN (WO2023012096A1) in view of BOURKE (US 20200334928 A1) in further view of ROTTNER (US 20130107048 A1).
Regarding claim 4:
IKEN in view of BOURKE discloses the limitation within claim 1 and IKEN further discloses:
according to existing images around the corresponding location on the image map; and (see at least IKEN, ¶ 0012, "Preferably, the high-resolution map is determined based on the map data and/or can be determined from it. For example, the map data can be characteristic of a variety of (image) information (such as intensity, color and/or brightness values) of the high-resolution map, especially at the pixel level of the map.")
IKEN in view of BOURKE does not disclose, but ROTTNER teaches:
the step of processing the images further includes adjusting a brightness of the written images based on one or any combination of the following: (see at least ROTTNER, ¶ 0002, “When a video camera system includes multiple cameras, the images received from the cameras can be combined or stitched together to produce one or more panoramic images. However, the individual images making up the panoramic image may have different image characteristics, such as contrast or brightness. Therefore, the resulting panoramic image may not provide a driver with a clear or consistent view of the areas surrounding the vehicle. Without a clear view, the viewer may not be able to safely or efficiently maneuver the vehicle. In addition, if other vehicle control systems, such as an object-detection system, use the panoramic image, errors or problems with the panoramic image may cause the vehicle control systems to malfunction or provide false alarms or alerts.”; ¶ 0018, “As noted above, the ECU 40 performs automatic image equalization. Image equalization is the process of adjusting the brightness of an image to improve its contrast. If image equalization is not performed, the images created from the image data received from the four cameras 14, 16, 18, and 20 may have varying contrasts and brightness, which make the resulting stitched images more difficult for an operator to view and interpret. FIG. 4 illustrates an image equalization method 60 performed by the ECU 40 according to one embodiment of the invention. The image equalization method 60 creates increased consistency in contrast and brightness between the image data received from the four cameras 14, 16, 18, and 20. For simplicity, one iteration of the logic of FIG. 4 is explained in detail below. However, it should be understood that the logic is executed rapidly by the ECU 40 and new image data is received by the ECU 40 and new panoramic images are displayed on the viewing screen 50 in a substantially continuous manner.”)
according to adjacent received images. (see at least ROTTNER, ¶ 0018, “As noted above, the ECU 40 performs automatic image equalization. Image equalization is the process of adjusting the brightness of an image to improve its contrast. If image equalization is not performed, the images created from the image data received from the four cameras 14, 16, 18, and 20 may have varying contrasts and brightness, which make the resulting stitched images more difficult for an operator to view and interpret. FIG. 4 illustrates an image equalization method 60 performed by the ECU 40 according to one embodiment of the invention. The image equalization method 60 creates increased consistency in contrast and brightness between the image data received from the four cameras 14, 16, 18, and 20. For simplicity, one iteration of the logic of FIG. 4 is explained in detail below. However, it should be understood that the logic is executed rapidly by the ECU 40 and new image data is received by the ECU 40 and new panoramic images are displayed on the viewing screen 50 in a substantially continuous manner.”; ¶ 0021, “Next, the ECU 40 adjusts the brightness of the image data received from the first camera 14, the second camera 16, and the third camera 18 (i.e., the front, left, and right cameras). As shown in FIG. 4, this adjustment includes determining a darkest region in the image data received from the first, second, and third cameras 14, 16, and 18 (at 66) and adjusting the brightness of the image data from these cameras based on the darkest region (at 68). In some embodiments, the ECU 40 determines a darkest region in the image data by comparing the pixel intensity of the stitching areas 30 and 32 generated when the image data from the three cameras 14, 16, and 18 is image stitched. In other embodiments, the ECU 40 determines a darkest region within the entire image data received from each of the three cameras 14, 16, and 18. Once the ECU 40 identifies the darkest region, the ECU 40 adjusts the brightness of the image data from the three cameras 14, 16, and 18 by comparing the pixel intensity value of the darkest region to pre-set image adjustment settings. The pre-set image adjustment settings can provide brightness adjustments based on the pixel intensity of the darkest region (e.g., in the form of a look-up table or a formula). The brightness adjustment can ensure that the overall image data is not too dark or too light where objects in extreme dark or bright regions are unidentifiable by the vehicle operator. Therefore, the pre-set image setting can increase or decrease the brightness of the image data from the three cameras 14, 16, and 18 to ensure that image data has a useful contrast. In some embodiments, the ECU 40 only adjusts the brightness of the image data in the stitching areas 30 and 32. In other embodiments, the ECU 40 adjusts the brightness of the entire image data received from each of the three cameras 14, 16, and 18.”)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify, with a reasonable expectation of success, the stitching of multiple cameras for mapping an environment accident detection wherein certain components crossing a threshold trigger a notification within IKEN in view of BOURKE to include the correction for brightness and contrast when stitching images within ROTTNER to effectively yield cleaner surround images of the vehicle for ensuring cleaner, more consistent mapping.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
YORIFUJI (US 20240192020 A1)
¶ 0005, “One aspect of the present disclosure is a map management apparatus that includes road information including a road shape and a road type and speed sign information and issues a notification for prompting update of map information stored in an in-vehicle device of an own vehicle, and the map management apparatus includes a recognition unit that recognizes at least any one of a road shape of a road where the own vehicle travels, a road type of the road where the own vehicle travels, and a speed sign provided on the road where the own vehicle travels, on the basis of a captured image of an in-vehicle camera that images surroundings of the own vehicle, a first determination unit that determines whether or not there is a difference between recognition information including information regarding at least any one of the road shape, the road type, and the speed sign recognized by the recognition unit and the map information, a second determination unit that determines whether or not there is a difference between a speed of the own vehicle and the speed sign information included in the map information, and a notification unit that issues a notification for prompting update of the map information, when the number of times when the first determination unit determines that there is the difference and the second determination unit determines that there is the difference becomes equal to or more than a predetermined determination frequency threshold.”
WANG (WO2022204855A1)
¶ 0040, "Determine the pose changes of M cameras on the target terminal, where M is an integer greater than 1; update the camera parameters of the M cameras based on the pose changes of the M cameras, the camera parameters including the mapping relationship between the coordinate system of the image captured by the corresponding camera and the world coordinate system; based on the updated camera parameters of the M cameras, stitch together the images captured by the M cameras to obtain a target image whose viewing angle in the world coordinate system exceeds a preset viewing angle threshold. "
¶ 0086, "For example, intelligent vehicles can use onboard integrated positioning systems or sensors such as GPS+IMU to determine the differences between images taken by multiple onboard cameras at the same location. This allows them to determine whether the camera parameters of the onboard cameras need to be corrected or optimized to ensure that changes in the vehicle's posture do not affect the shooting status of its onboard cameras, nor the generation of images of the surrounding environment. "
CARRUTHERS (US 20210097315 A1)
¶ 0051, “The data capture functionality 240 may provide functionality for capturing data from numerous sources, including the forward camera 212 and rear camera 214. The data capturing functionality 240 may control details of how the data is captured, including for example, frame rates, resolutions, encoding and compression. The data capture functionality 240 may also capture data from other sensors such as the GPS component 222 and the accelerometers and gyroscopes 224. The video data may be stored in short video segments, such as for example between 30 seconds to 5 minutes in length, in a rolling window. That is, newer video shorts may overwrite older video shorts, unless the older video short has been flagged for longer storage or transfer. The particular data captured from the various data sources may be stored in association with time information to allow the stored data to be combined together at a later time. For example, video and audio data can be correlated with logged GPS data based on the time the data was captured to determine the location that the particular audio/video was captured. Additionally, or alternatively, sensor data may be combined with the audio/video data as it is captured and stored. For example, GPS information, speed information or other captured data, may be overlaid on the captured video data as it captured and stored. Additionally, the data capture functionality 240 may periodically capture, such as for example between every 1 to 5 seconds, still images from the front camera and/or rear camera to provide a time-lapse video of a trip. The time-lapse may be captured for a particular trip, which may be specified by the user, or may be determined automatically by the in-vehicle monitoring device. The in-vehicle device may determine a single trip to begin with turning on the vehicle and may end with turning off the vehicle. A trip may continue after turning off the vehicle if the vehicle is turned back on within a particular length of time, such as for example 5 minutes to 10 minutes.”
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAFAEL VELASQUEZ VANEGAS whose telephone number is (571)272-6999. The examiner can normally be reached M-F 8 - 4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, VIVEK KOPPIKAR can be reached at (571) 272-5109. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAFAEL VELASQUEZ VANEGAS/Patent Examiner, Art Unit 3667
/VIVEK D KOPPIKAR/Supervisory Patent Examiner
Art Unit 3667
December 1, 2025
