DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim(s) 9 is/are objected to because of the following informalities:
In claim 9, line 8, “[…] a height of the object nd/or of at” should read ““[…] a height of the object and/or of at”.
Appropriate correction is required.
Office Action Summary
Claim(s) 1-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rueegg et al (DE 10 2020 209 218 A1; See translation provided by Examiner) in view of Adel et al (DE 10 2017 101 476 B3; See translation provided by Examiner).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rueegg et al (DE 10 2020 209 218 A1; See translation provided by Examiner) in view of Adel et al (DE 10 2017 101 476 B3; See translation provided by Examiner).
Regarding claim(s) 1, 11, and 12, Rueegg teaches a non-transitory computer-readable storage medium on which are stored instructions for processing data associated with at least one ultrasonic sensor, the instructions, when executed by a computer (Paragraph [0003]), causing the computer to perform the following steps:
receiving at least one signal characterizing at least a portion of a transmitted ultrasonic signal (Paragraph [0005]: “for at least one detection pair, each consisting of an ultrasound transmitter and an ultrasound receiver of the mobile platform […] at least one ultrasound signal received by the respective ultrasound receiver, wherein the received ultrasound signal was reflected from the environment of the mobile platform […]”);
determining, based on the at least one signal, a two-dimensional data set representing a plurality of cells characterizing (Paragraph [0005]: “accessible grid cells of the spatial grid formed around the mobile platform are determined by […] detection pair on the mobile platform and at least one ultrasound path, and a machine learning method set up and trained for this purpose, wherein the at least one ultrasound path is determined by means of the at least one ultrasound transit time”; Paragraph [0021]: “the ultrasound signals […] corresponding to a two-dimensional image in which at least one value is assigned to each pixel”; and Paragraph [0025]: “[…] the spatial grid has two dimensions and is arranged horizontally to the mobile platform in such a way that the accessible grid cells of the two-dimensional spatial grid characterize the traversability of the accessible grid cells by the mobile platform”); and
processing the data set using an artificial neural network (Paragraph [0009]: “The machine learning method can be implemented using a trained neural convolutional network, which may be structured in combination with fully connected neural networks […]”).
Rueegg fails to teach to determining, based on the at least one signal, a two-dimensional data set representing a plurality of cells characterizing potential positions of an object relative to the ultrasonic sensor associated with the at least one signal. However, Adel teaches to determining, based on the at least one signal, a two-dimensional data set representing a plurality of cells characterizing potential positions of an object relative to the ultrasonic sensor associated with the at least one signal (Figure 2; Paragraph [0032]: “wherein each grid cell of the occupancy grid map corresponds to a predetermined area in the environment. The weight of each grid cell represents a probability for the corresponding predetermined area in the environment to be occupied”; and Paragraph [0044]: “the processing unit 4 is configured to increase the weights of the grid cells 7a by a predetermined value representing a range with a distance from the ultrasonic sensor 3a' to 3l' corresponding to the calculated distance d1, d2. This leads to so-called ultrasonic arcs 11a to 11e (FIG. 2) which are circle segments with respective distances d1, d2 from the ultrasonic sensor 3a' to 3l'”); and
processing the data set using an artificial neural network (Figure 3; and Paragraph [0033]: “to use the normalized occupancy grid map or the respective part of the occupancy grid map as an input for a multilayer neural network for object tracking and to calculate a new occupancy grid map based on the output of the neural network”).
Rueegg teaches receiving ultrasonic signals, determining a two-dimensional spatial grid including a plurality of grid cells based on the received ultrasonic signals, and processing the spatial-grid data using a machine learning method, including an artificial neural network, to determine object-related information. However, Rueegg does not explicitly teach representing potential positions of an object by updating the values of the plurality of cells corresponding to the possible object locations associated with ultrasonic measurements. Adel teaches constructing a weighted occupancy-grid map in which each grid cell corresponds to a predetermined area of the environment, wherein the weight of each grid cell represents a probability that the corresponding area is occupied, and further teaches calculating a distance corresponding to a detected ultrasonic echo and increasing the weights of the grid cells corresponding to the calculated distance to generate ultrasonic arcs representing possible object locations before processing the occupancy-grid map using a multilayer neural network.
Therefore, it would have been obvious to one of ordinary skill in the art to combine Rueegg and Adel before the effective filing date of the claimed invention. The motivation for this combination of references would have been to improve the quality of the occupancy grid map and reduce the uncertainty in the detected objects generally located on ultrasonic arcs by processing the occupancy grid map using a multilayer neural network. This motivation for the combination of Rueegg and Adel is/are supported by KSR exemplary rationale (G) Some teaching, suggestion, or motivation in the prior art that would have led one of ordinary skill to modify the prior art reference or to combine prior art reference teachings to arrive at the claimed invention. MPEP 2141 (III).
Regarding claim(s) 2, Rueegg as modified by Adel teaches the method according to claim 1, wherein the data set represents a rectangular array of the cells corresponding to respective relative positions with respect to the ultrasonic sensor, wherein each of the cells may be assigned at least one cell value based on the at least one signal (where Rueegg teaches in Paragraph [0021]: “the ultrasound signals […] corresponding to a two-dimensional image in which at least one value is assigned to each pixel”; and where Adel teaches in Paragraph [0008]: “Each grid cell of the occupancy grid map corresponds to a predetermined area in the environment. The predetermined area in the environment may have a rectangular shape […] The weight or weight of each grid cell represents a (corresponding to) probability for the corresponding predetermined area in the environment to be occupied […]”; and Paragraph [0010]: “The weights of all grid cells that represent an area in the surroundings of a motor vehicle with the calculated distance from the ultrasonic sensor can be increased”).
Regarding claim(s) 3, Rueegg as modified by Adel teaches the method according to claim 1, wherein the neural network is a convolutional neural network of a U-Net type, wherein the neural network is configured to receive image data as input data (where Rueegg teaches in Paragraph [0009]: “The machine learning method can be implemented using a trained neural convolutional network, which may be structured in combination with fully connected neural networks, possibly using classical regularization and stabilization layers such as batch normalization and training dropouts, and using various activation functions such as sigmoid and ReLu […]”; and Paragraph [0021]: “the ultrasound signals […] corresponding to a two-dimensional image in which at least one value is assigned to each pixel”; and where Adel teaches in Paragraph [0045]: “The input layer 9a corresponds to the normalized occupancy grid map 6b and is connected to an encoder layer 9b […] The conjecture tracking layer 9c projects onto a decoder or decoder layer 9d […]”).
Regarding claim(s) 4, Rueegg as modified by Adel teaches the method according to claim 1, where Adel teaches further comprising:
determining at least one echo based on the at least one signal (Paragraph [0009]: “at least one ultrasonic signal […] of the ultrasonic sensor system […] detecting at least one echo […] of the at least one ultrasonic sensor by at least one ultrasonic sensor […] Another method step is calculating at least one distance, i.e. one or more distances, from the at least one ultrasonic sensor system”);
determining a curve associated with the at least one echo (Paragraph [0012]: “detected object that has generally been located on an ultrasonic arc. In this case, the ultrasonic arc is given or determined by a time difference between the emission and the detection of the echo”); and
modifying the data set based on the curve (Paragraph [0010]: “The weights of all grid cells that represent an area in the surroundings of a motor vehicle with the calculated distance from the ultrasonic sensor can be increased”).
Regarding claim(s) 5, Rueegg as modified by Adel teaches the method according to claim 4, where Adel teaches further comprising:
mapping the curve to the plurality of cells and modifying a cell value of a respective cell of the cells based on the mapping (Paragraph [0043]: “[…] a weighted occupancy grid map 6a, wherein each grid cell 7a of the occupancy grid map 6a corresponds to a predetermined area in the environment 5 and the weighting of each grid cell 7a represents a probability for the corresponding predetermined area in the environment 5 to be occupied”; and Paragraph [0044]: “the processing unit 4 is configured to increase the weights of the grid cells 7a by a predetermined value representing a range with a distance from the ultrasonic sensor 3a' to 3l' corresponding to the calculated distance d1, d2. This leads to so-called ultrasonic arcs 11a to 11e (FIG. 2) which are circle segments with respective distances d1, d2 from the ultrasonic sensor 3a' to 3l' […]”).
Regarding claim(s) 6, Rueegg as modified by Adel teaches the method according to claim 1, where Rueegg teaches further comprising:
determining a feature based on the at least one signal (Paragraph [0032]: “at least one received ultrasound signal, a signal amplitude is determined […] The structure of the input data […] can then be extended […] by assigning a signal amplitude […]”; and Paragraph [0039]: “a signal amplitude of the received ultrasound signal and/or a quality of agreement between the received and the emitted ultrasound signal and/or a frequency shift between the emitted and the received ultrasound signal is provided”);
modifying a cell value of a respective cell of the cells based on the feature (Paragraph [0032]: “[…] using the signal amplitude, at least one ultrasound path is determined with a machine learning method […] the method for determining accessible grid cells can be improved, since the signal amplitude provides information about the environment of the correspondingly determined grid cell”; and Paragraph [0031]: “[…] the goodness of fit provides information about the environment of the correspondingly determined grid cell”).
Regarding claim(s) 7, Rueegg as modified by Adel teaches the method according to claim 6, wherein the feature includes at least one of:
a) an amplitude of at least one echo associated with the at least one signal, or b) an average background noise associated with the at least one signal, or c) a number of echoes associated with the at least one signal (where Rueegg teaches in Paragraph [0032]: “at least one received ultrasound signal, a signal amplitude is determined, and using the signal amplitude, at least one ultrasound path is determined […] the signal amplitude provides information about the environment of the correspondingly determined grid cel”; and Paragraph [0039]: “a signal amplitude of the received ultrasound signal and/or a quality of agreement between the received and the emitted ultrasound signal and/or a frequency shift between the emitted and the received ultrasound signal is provided”; and where Adel teaches in Paragraph [0012]: “[…] when multiple ultrasonic echoes and/or multiple ultrasonic signals are used, the neural network is particularly well suited to finding hidden relationships within the sensor data. The neural network may find out hidden relationships in the detected echoes and the increased weights of the grid cells […]”; and Paragraph [0055]: “a trained network 8 is able to clean up noise, detect an object 10 (Fig. 1) and perform correct object reconstruction when there are a plurality of echoes […]”).
Regarding claim(s) 8, Rueegg as modified by Adel teaches the method according to claim 1, further comprising:
where Rueegg teaches providing the data set in a form of one or more layers to the neural network (Table 1; Table 2; Paragraph [0020]: “the input data for the trained machine learning algorithm can be structured according to the following table for the respective detection pairs”; Paragraph [0032]: “at least one received ultrasound signal, a signal amplitude is determined […] The structure of the input data […] can then be extended […] by assigning a signal amplitude […]”; and Paragraph [0039]: “a signal amplitude of the received ultrasound signal and/or a quality of agreement between the received and the emitted ultrasound signal and/or a frequency shift between the emitted and the received ultrasound signal is provided”), and where Adel teaches wherein each layer of the one or more layers represents the plurality of cells and corresponding cell values associated with at least one feature determined based on the at least one signal (Paragraph [0043]: “[…] a weighted occupancy grid map 6a, wherein each grid cell 7a of the occupancy grid map 6a corresponds to a predetermined area in the environment 5 and the weighting of each grid cell 7a represents a probability for the corresponding predetermined area in the environment 5 to be occupied”; Paragraph [0045]: “The input layer 9a corresponds to the normalized occupancy grid map 6b and is connected to an encoder layer 9b […] The conjecture tracking layer 9c projects onto a decoder or decoder layer 9d […]”; and Paragraph [0044]: “the processing unit 4 is configured to increase the weights of the grid cells 7a by a predetermined value representing a range with a distance from the ultrasonic sensor 3a' to 3l' corresponding to the calculated distance d1, d2”).
Regarding claim(s) 9, Rueegg as modified by Adel teaches the method according to claim 1, where Rueegg teaches further comprising:
training the neural network to provide, as output data, a two-dimensional data set representing the plurality of cells (Paragraph [0009] – [0013]; Paragraph [0019]: “adequately describes the environment of a mobile platform into accessible and inaccessible grid cells of a spatial grid […]”; and Paragraph [0026]: “the spatial grid can be limited to two dimensions. This simplification is aided by the fact that, for example, relevant objects in road traffic have ground contact and height information is of little relevance if such an object […]”), wherein each cell of the cells is associated with at least one of:
where Adel teaches a) a first output value characterizing a probability for a presence of the object and/or at least one further object (Paragraph [0043]: “[…] a weighted occupancy grid map 6a, wherein each grid cell 7a of the occupancy grid map 6a corresponds to a predetermined area in the environment 5 and the weighting of each grid cell 7a represents a probability for the corresponding predetermined area in the environment 5 to be occupied”; and Paragraph [0044]: “the processing unit 4 is configured to increase the weights of the grid cells 7a by a predetermined value representing a range with a distance from the ultrasonic sensor 3a' to 3l' corresponding to the calculated distance d1, d2. This leads to so-called ultrasonic arcs 11a to 11e (FIG. 2) which are circle segments with respective distances d1, d2 from the ultrasonic sensor 3a' to 3l' […]”), or where Rueegg teaches b) a second output value characterizing a height of the object and/or of at least one further object (Paragraph [0026]: “the spatial grid can be limited to two dimensions. This simplification is aided by the fact that, for example, relevant objects in road traffic have ground contact and height information is of little relevance if such an object […]”; and Paragraph [0027]: “the environment is described using a 2.5-dimensional world, i.e., a two-dimensional world with additional height information”).
Regarding claim(s) 10, Rueegg as modified by Adel teaches the method according to claim 1, where Rueegg teaches further comprising:
receiving a plurality of signals characterizing at least a portion of a respective plurality of transmitted ultrasonic signals (Table 1; and Paragraph [0005]: “for at least one detection pair, each consisting of an ultrasound transmitter and an ultrasound receiver of the mobile platform […] at least one ultrasound signal received by the respective ultrasound receiver, wherein the received ultrasound signal was reflected from the environment of the mobile platform […]”); and
determining the two-dimensional data set based on the plurality of signals (Table 1; Paragraph [0020]: “the input data for the trained machine learning algorithm can be structured according to the following table for the respective detection pairs”; and Paragraph [0021]: “the ultrasound signals […] corresponding to a two-dimensional image in which at least one value is assigned to each pixel”).
Regarding claim(s) 13, Rueegg as modified by Adel teaches the method according to claim 1, where Rueegg teaches wherein the method is used for at least one of:
a) processing at least one signal characterizing at least a portion of a transmitted ultrasonic signal, or b) using a neural network to determine a position of an object, or c) using a neural network to determine a height of an object, or d) assisting a parking procedure of a vehicle, or e) triggering an emergency brake of a vehicle (Paragraph [0005]: “for at least one detection pair, each consisting of an ultrasound transmitter and an ultrasound receiver of the mobile platform […] at least one ultrasound signal received by the respective ultrasound receiver, wherein the received ultrasound signal was reflected from the environment of the mobile platform […]”; Paragraph [0046]: “[…] the received ultrasound signal was reflected from the environment of the mobile platform from inaccessible grid cells A, B […] accessible grid cells of the spatial grid 130, i.e. all grid cells 130 except A and B, are determined by means of the arrangement of the at least one detection pair 121, 122 of the mobile platform 110 and at least one ultrasound path, and a machine learning method […] The spatial grid 130, which has both accessible and inaccessible grid cells, is formed around the mobile platform 110 in such a way that it moves with the mobile platform”; Paragraph [0026]: “the spatial grid can be limited to two dimensions. This simplification is aided by the fact that, for example, relevant objects in road traffic have ground contact and height information is of little relevance if such an object […]”; Paragraph [0027]: “the environment is described using a 2.5-dimensional world, i.e., a two-dimensional world with additional height information”; Paragraph [0001]: “[…] this data has mainly been used to distinguish between free space and the presence of possible obstacles, for example for parking maneuvers”).
Relevant Prior Art Directed to State of Art
Kober et al (US 2017/0067855 A1) are relevant prior art not applied in the rejection(s) above. Kober discloses a method for testing a workpiece using ultrasound in a curved area of the surface of said workpiece, having the following steps: (a) a plurality of ultrasonic signals are emitted from a plurality of transmitting positions under different pivot angles lying in a pivoting range by means of at least one ultrasonic transducer and are injected into the workpiece, (b) a corresponding ultrasonic echo signal is received for each ultrasonic signal and the amplitude of the ultrasonic echo generated upon entering the workpiece or on the rear wall of the workpiece is determined, (c) for each transmitting position the ultrasonic echoes having amplitudes representing local maxima are determined, (d1) if a single ultrasonic echo having an amplitude representing a local maximum has been determined for a transmitting position in step (c), the associated ultrasonic echo signal of said echo is selected, (d2) if in step (c) a plurality of ultrasonic echoes having an amplitude representing a local maximum have been determined for a transmitting position, or if this is predefined for a transmitting position, a selection of ultrasonic echo signals is made, provided only a single ultrasonic echo having an amplitude representing a local maximum has been determined in step (c) for an adjacent transmitting position, by selecting those ultrasonic echo signals which lie in a specific angle range around the corresponding pivot angle of the ultrasonic echo signal having the maximum amplitude of the ultrasonic echo of the adjacent transmitting position and which have an ultrasonic echo having a maximum amplitude, (e) an evaluation is carried out of at least the selected ultrasonic echo signals.
Weikersdorfer et al (US 2024/0176017 A1) are relevant prior art not applied in the rejection(s) above. Weikersdorfer discloses a method comprising: receiving sensor data generated using an ultrasonic sensor of a machine; generating, using one or more neural networks and based at least on the sensor data, at least one of a height map or an occupancy map; and performing one or more operations based at least on the at least one of the height map or the occupancy map. Further comprising: generating, based at least on the sensor data, input data representing one or more locations of one or more objects, wherein the generating the at least one of the height map or the occupancy map is based at least on the input data, wherein the generating the input data representing the one or more locations of the one or more objects comprises: determining, based at least on the sensor data, that one or more amplitudes associated with one or more bins are equal to or greater than a threshold amplitude; determining that the one or more bins are associated with one or more distances to the one or more objects; determining the one or more locations based at least on the one or more distances; and generating the input data to represent the one or more locations.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JONGBONG NAH whose telephone number is (571) 272-1361. The examiner can normally be reached M - F: 9:00 AM - 5:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, ONEAL MISTRY can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JONGBONG NAH/Examiner, Art Unit 2674
Prosecution Insights