DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 16-29 are presented for examination.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5).
Claims 16 and 23 are objected to because of minor informalities.
Claims 19, 26 and 27 are rejected under 35 U.S.C. 112(b)
Claims 16-29 are not found eligible under 35 USC 101.
Claims 16, 25, 28-29 are rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1) further in the view of He; Fangning (US10809073B2).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1) further in the view of He; Fangning (US10809073B2) further in the view of Pink; Oliver (US9823661B2).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1) further in the view of He; Fangning (US10809073B2) further in the view of Pink; Oliver (US9823661B2) further in the view of Korte; Theodore H. (US11025972B2).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1) further in the view of Korte; Theodore H. (US11025972B2)
Claims 22 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1)further in the view of Chavali; Pothuraju (US 20190228237 A1).
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1)further in the view of Chavali; Pothuraju (US 20190228237 A1) further in the view of Haitsma; Jaap Andre (US 7549052 B2).
Claims 26 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1)further in the view of Dolan; James Graham (US 11415997 B1).
This action is Non-Final rejection.
Priority
Acknowledgment is made for applicants claimed foreign priority of application DE 102020215535.6 filed on 12/09/2020.
Information Disclosure Statement
The IDS filed on 06/07/2023, 08/31/2023 and 07/24/2025 is reviewed and considered. See the attached document.
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description [0042] as reference number 50 vehicle is not shown on Fig. 1. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Objections
Claims 16 and 23 are objected to because of the following informalities:
Claim 16, The first and second digital representation should be introduced after the preamble, since the preamble only introduce two digital representation without introducing as a first and second.
Claim 23, “filtering core” should be an act of filtering, if filtering core is a module, it is not well-known filtering algorithm.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 19, 26 and 27 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
The term “the greater the importance” in claim 19 is a relative term which renders the claim indefinite. The term “the greater the importance” is not defined by the claim, the specification does not provide a standard for ascertaining the requisite degree, and one of ordinary skill in the art would not be reasonably apprised of the scope of the invention. The word “the greater the importance” is very subjective, so it brings the claim undefine.
Claim 26 recites the limitation
“the reference driving situation” in line 8
There is insufficient antecedent basis for this limitation in the claim.
Claim 27 is rejected under the same rational since it is dependent on claim 26.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 29 is rejected under 35 U.S.C. 101 because the claimed invention is
directed to non-statutory subject matter. The claims does not fall within at least one of
the four categories of patent eligible subject matter because both claim 29 is a
computer without any hardware component so it is considered as a program which is directed to software per se because only if at least one of the claimed elements of the method or system is a physical part of a device can the method as claimed constitute part of a device or combination of devices to be a machine within the meaning of 101. Since a “A computer” consists merely instruction and the Claim 29 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claims 16-29 are rejected under 35 U.S.C. 101 because the claim invention
recites a judicial exception, which is directed to judicial exception of an abstract idea, as
it has not been integrated into practical application and the claim further do not recite
significantly more that the judicial exception.
Step 1: claims 16-27 are directed to a method, which is a process, which is
a statutory category of invention. Claims 28 are directed a machine, which is a statutory category of invention. While claim 29 is directed to a computer program, which is not a statutory category of invention as discussed above as software per se.
Step 2A, Prong 1: Yes, the claims recites abstract idea . Claims 16, 26, 28-29 recites abstract idea under mental process. A human mind which can be perform comparing of two digital representation of a driving situation with the aid of pencil and paper. Abstract ideas are bolded as shown below.
Claim 16, 26, 28 - 29: recites
subdividing a region of the environment of the vehicle into a grid of subregions, under its broadest reasonable interpretation, a human mind can divide the driving environment using a pen and paper into a subregions since there is no specific way of dividing or using of software is claimed. Therefore it is an abstract idea under mental process.
ascertaining, based on the occupancy information in the first digital representation, occupancies of the subregions by traffic-relevant objects, and combining them to form a first fingerprint of the first driving situation: under its broadest reasonable interpretation, this claim limitation can also be performed by a human mind through observation, analysis and judgment. A human mind can observe the driving environment’s subregion to check if the subregion contains any traffic relevant object and can create a digital representation (fingerprint) based on the observed driving environment. Therefore it is an abstract idea under mental process. A claim to "collecting information, analyzing it, and displaying certain results of the collection and analysis," where the data analysis steps are recited at a high level of generality such that they could practically be performed in the human mind, Electric Power Group v. Alstom, S.A., 830 F.3d 1350, 1353-54, 119 USPQ2d 1739, 1741-42 (Fed. Cir. 2016);
ascertaining, based on the occupancy information in the second digital representation, occupancies of the subregions by traffic-relevant objects, and combining them to form a second fingerprint of the second driving situation; under its broadest reasonable interpretation, this claim limitation can also be performed by a human mind through observation, analysis and judgment. A human mind can observe the driving environment’s subregion to check if the subregion contains any traffic relevant object and can create a digital representation (fingerprint) based on the observed driving environment. Therefore it is an abstract idea under mental process.
ascertaining a similarity measure between the first fingerprint and the second fingerprint according to a predefined measure specification; under its broadest reasonable interpretation , this claim limitation can be performed by a human mind through observation, analysis and judgment. A human mind can compare two digital representations by observing according to predetermined specification. Therefore it is an abstract idea under mental process. A claim to collecting and comparing known information (claim 1), which are steps that can be practically performed in the human mind, Classen Immunotherapies, Inc. v. Biogen IDEC, 659 F.3d 1057, 1067, 100 USPQ2d 1492, 1500 (Fed. Cir. 2011);
determining that the first and second driving situations are identical or at least similar based on the similarity measure satisfying a predefined criterion: this claim limitations can also be performed by a human mind through observation, analysis and judgment. By observing and analyzing the two-driving situation, a human mind can make a judgment about their similarity based on the predefined criterion. Therefore it is an abstract idea under mental process. A claim to collecting and comparing known information (claim 1), which are steps that can be practically performed in the human mind, Classen Immunotherapies, Inc. v. Biogen IDEC, 659 F.3d 1057, 1067, 100 USPQ2d 1492, 1500 (Fed. Cir. 2011);
Claim 26: in addition to the above abstract ideas, it also recites
converting at least one simulated driving situation of a simulated drive into a first digital representation; under a broadest reasonable interpretation, this limitation recites a mental process, since this can be done by a human mind by the aid of pen and pencil. The limitation doesn’t use any specific computer implemented step or software to convert the simulated driving situation into a digital representation, so a human mind can create a digital representation by observing analyzing, evaluating and making judgment on the simulated environment.
ascertaining second digital representations of a multitude of reference drives within the reference drives from time characteristics of reference drives; this claim limitation recites a mental process since this limitation can be done by a human mind. A human mind can create another digital representation from the reference drive situation. A human can observe the reference drive and create a digital representation.
comparing the first digital representation of the simulated driving situation against each of the second digital representations of the reference driving situations, the first and second digital representations including occupancy information about an occupancy of the environment of the vehicle by traffic-relevant objects, this claim limitation recites a mental process, since a human mind can compare two digital representations by preforming observation on each of the occupancy information about the environment. This limitation does not recite any specific way of comparing or using software, a human mind can make a judgment by observing both digital representation.
ascertaining from results of the comparisons at least one reference driving situation within a reference drive that is identical or at least similar to the simulated driving situation; this claim limitation also recites a mental process since a human mind can make a judgment based the result of the comparation whether a simulated driving situation matches at least one reference driving situation.
comparing a characteristic of the simulated drive following the simulated drive situation with a characteristic of the reference drive following the ascertained reference driving situation; it is also a mental process, since a human mind can compare two digital representation by preforming observation on each of the occupancy information about the environment. This claim is just comparing of two driving situation and a human mind is capable of performing a comparation by performing observation, evaluation, analysis and judgment.
determining that the simulation of the drive is valid at least for the simulated driving situation when the following characteristic of the simulated drive is in line according to the predefined criterion with the following characteristic of the reference drive: under a broadest reasonable interpretation this claim recites a mental process. The Claim limitation does not recite any specific way of checking validity and performing simulation, so this can be performed by a human mind by comparing the expected values defined in advance with the ideal performance of the vehicle through observation, evaluation, analysis and judgement.
Step 2A prong 2: No
The above judicially exceptions do not recite additional elements that integrate
the exceptions into a practical application of the exception because the claims do not
have additional elements of a combination of additional elements that apply, rely or use
the judicial exception in a manner that impose a meaningful limit on the judicial
exception. As it is analyzed above the claims recites comparing two digital representation without any additional elements which integrated the abstract idea of comparing a digital representation into a practical application.
Step 2B :No:
The claims do not cite additional elements which are significantly more
than the abstract idea. As outlined above the claims merely use a computer to perform
abstract ideas. Merely using of a computer and applying abstract ideas into a system
without making improvement to the functionality of a computer is not a significantly
more.
Claims 28 recites “non-transitory machine-readable data carrier on
which is stored a computer program” interpreted as additional element and it is used as a tool to perform the abstract idea of comparing two digital representation using a computer or software as a tool. As it is claimed in the claims there is no improvement to the computer or software is recited, so it is not significantly more.
Generally the independent claims inherently recites abstract idea based on the above analysis and let’s see if there is any significant more claim limitations exist for the
dependent claims.
Claim 17:
wherein it is binarily detected respectively in one bit in each case whether a subregion is occupied by at least one traffic-relevant object: it further defines the abstract idea of assigning a binary for a region is occupied or not, so other than narrowing the abstract idea, no new additional element is recited which is significantly more.
Claim 18:
wherein the bits acquired for all subregions are combined in a binary number as a fingerprint – it further defines abstract idea , a human mind can combine the binary numbers by making observation on the subregions, so other than narrowing the abstract idea, no new additional element is recited which is significantly more.
Claim 19:
wherein a significance of a bit in the binary number is lower or higher the
greater the importance of an object in the corresponding subregion for a behavior strategy of the vehicle in the first or second driving situation – this also further defines abstract idea since a human mind can assign the bits based on the importance of the object in the subregion by making a judgment, so other than narrowing the abstract idea, no new additional element is recited which is significantly more.
Claim 20:
wherein the similarity measure is a function of a length of a bit sequence that is identical in both the first and second fingerprints – it further defines the abstract idea of a similarity measurement, so other than narrowing the abstract idea, no new additional element is recited which is significantly more.
Claim 21:
wherein the similarity measure is a function of a Hamming distance between the first and second fingerprints – this claim limitation is also specifying the type of similarity measurement used, so other than narrowing the abstract idea, no new additional element is recited which is significantly more.
Claim 22:
wherein the occupancies acquired for all subregions are combined in a matrix in each case and the first and second fingerprints are formed therefrom – it further defines abstract idea and it can be done by a human mind by using a pen and paper, no new additional element is recited which is significantly more.
Claim 23:
wherein at least the matrix generated from one of the first or second digital representations is converted into the first or second fingerprint with the aid of filtering using at least one filter core - it further defines abstract idea and it can be done by a human mind by using a pen and paper, since a human mind can apply filter on a matrix to create a digital representation like tuple or array, so no new additional element is recited which is significantly more.
Claim 24:
wherein the similarity measure includes an average value or median of
elements in an elementwise product of both the first and second fingerprints – this claim limitation is also specifying the type of similarity measurement used, so other than narrowing the abstract idea, no new additional element is recited which is significantly more.
Claim 25:
wherein the first digital representation is ascertained from at least one simulation
of a driving situation and the second digital representation is ascertained from at least one recording of measured data recorded using at least one sensor during a drive of a vehicle – this claim limitation further defines where the two digital representations as ascertained, so other than defining the type of data used, no new additional element is recited which is significantly more.
Claim 27:
wherein brake danger parameters and/or required steering potentials reached in the following characteristics of the simulated and the reference drive to mitigate the danger are used for the comparison of the following characteristics – it further defines abstract idea of comparing the parameter with threshold and this can be done by a human mind sing pen and paper, so no new additional element is recited which is significantly more.
Generally, based on the above claim by claim analysis and claims as a whole, it does not recite any additional element which is significantly more than the claimed invention. The claim is merely a mental process of comparing two digital representation using a computer and merely using a computer is not significantly more than abstract idea since there is not improvement in the computer recited.
Therefore claims 16-29 is not found eligible under 35 USC 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 16, 25, 28-29 are rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1).
As of claim 16, Atsmon teaches A method for comparing two digital representations of driving situations of a vehicle, (page 13 line 6- 11, In computing, a statistical fingerprint of identified data is a statistical value computed using the identified data that uniquely identifies the identified data for practical comparison purposes. The present invention additionally proposes in some embodiments thereof classifying the realism of the simulated driving environment using a difference between a simulation statistical fingerprint computed using simulated driving data collected from a simulated driving, and a real statistical fingerprint computed using the real traffic data)
the digital representations including occupancy information about an occupancy of the environment of the vehicle by traffic-relevant objects, the method comprising the following steps (Page 12, line 24- 31, Instead of randomly generating driving characteristics of a simulated agent, or randomly generating a simulated agent’ s behavior at every point of a simulation, the present invention, in some embodiments thereof, proposes learning realistic moving object characteristics and realistic movement patterns from real traffic data collected from real traffic environments. In such embodiments the present invention further proposes generating one or more simulated agents according to the learned moving object characteristics and realistic movement patterns, such that each simulated agent embodies a realistic type of moving object (that is, vehicle or pedestrian) and a realistic type of movement pattern in a realistic traffic environment).
combining them to form a first fingerprint of the first driving situation; (page 6, line 13 -14, The real statistical fingerprint is computed using the plurality of environment values and the plurality of agent values)
combining them to form a second fingerprint of the second driving situation; (page 6, line 21- 23, the simulation statistical fingerprint is computed using the plurality of simulation environment attribute values and the plurality of simulation agent values).
ascertaining a similarity measure between the first fingerprint and the second fingerprint according to a predefined measure specification; and (Page 8, line 13 – 16, modifying a plurality of other model parameters of the other simulation generation model to minimize another difference between another simulation statistical fingerprint, computed using the other simulated driving data, and the real statistical fingerprint, computed using the real input data…Page 13 line 8- 11, classifying the realism of the simulated driving environment using a difference between a simulation statistical fingerprint computed using simulated driving data collected from a simulated driving, and a real statistical fingerprint computed using the real traffic data).
determining that the first and second driving situations are identical or at least similar based on the similarity measure satisfying a predefined criterion (Page 13 line 6- 11, In computing, a statistical fingerprint of identified data is a statistical value computed using the identified data that uniquely identifies the identified data for practical comparison purposes. The present invention additionally proposes in some embodiments thereof classifying the realism of the simulated driving environment using a difference between a simulation statistical fingerprint computed using simulated driving data collected from a simulated driving, and a real statistical fingerprint computed using the real traffic data… Page 27 line 5 – 8, A realism score of a refiner measures how realistic a refined image generated by the refiner in response to an input image appears. In 812, processing unit optionally computes a plurality of realism scores, each indicative of a quality of realism of one of the plurality of refined output images. Optionally each of the plurality of realism scores is normalized in a range from 0 to 1). Realism is used as a predefined criterion to check its similarity.
Atsmon does not explicitly teach subdividing a region of the environment of the vehicle into a grid of subregions; ascertaining, based on the occupancy information in the first digital representation, occupancies of the subregions by traffic-relevant objects ascertaining, based on the occupancy information in the second digital representation, occupancies of the subregions by traffic-relevant objects.
While Peake teaches subdividing a region of the environment of the vehicle into a grid of subregions; (Abstract, an occupancy grid generator is used to generate an occupancy grid indicative of an environment of an autonomous vehicle from an imaging scene that depicts the environment…[0019] In some embodiments, the perception signals 508 include data representing “occupancy grids” (e.g., one grid per T milliseconds), with each occupancy grid indicating object positions (and possibly object boundaries, orientations, etc.) within an overhead view of the autonomous vehicle's environment).
ascertaining, based on the occupancy information in the first digital representation, occupancies of the subregions by traffic-relevant objects, and( [0119], In some embodiments, the perception signals 508 include data representing “occupancy grids” (e.g., one grid per T milliseconds), with each occupancy grid indicating object positions (and possibly object boundaries, orientations, etc.) within an overhead view of the autonomous vehicle's environment. Within the occupancy grid, each “cell” (e.g., pixel) may be associated with a particular class as determined by the classification module 512, possibly with an “unknown” class for certain pixels that were not successfully classified).based on the occupancy grids it shows the traffic relevant object which is included in perception signal so this is interpreted as the first digital representation.
ascertaining, based on the occupancy information in the second digital representation, occupancies of the subregions by traffic-relevant objects, and ([0119], Similarly, the prediction signals 522 may include, for each such grid generated by the perception component 506, one or more “future occupancy grids” that indicate predicted object positions, boundaries and/or orientations at one or more future times (e.g., one, two, and five seconds ahead). Occupancy grids are discussed further below in connection with FIGS. 6A and 6B). perception signal 522 including the future occupancy is considered as the second digital representation.
Atsmon and Peake is considered as analogous to the claimed invention since they focus on training vehicles using simulation realist data. Therefore it would be obvious for a person of ordinary skill in the art, before the effective filling date to determine a similarity using a fingerprint of an occupancy map and set a similarity threshold by combining Atsmon’s teaches of actual and simulated fingerprint to determine the similarity between a real environment and a simulated driving environment, and Peake’s teaching of occupancy grid and determining traffic relevant objects in subregions of the environment.
The motivation would have been to improve the efficiency and effectiveness of generating and/or collecting numerous autonomous driving datasets, and also address safety concerns with respect to generating sufficient datasets in a non-dangerous and controlled manner when training autonomous vehicles in real-world driving applications (Peake, [0006]).
Claim 28 and 29 is in the same scope as claim 16, with additional elements that Astmon teaches A non-transitory machine-readable data carrier on which is stored a computer program, and a computer (claim 29) (page 17 line 23-26, The present invention may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium ( or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention).
Therefore claims 28 and 29 are rejected under the same rational as of claim 16.
As of claim 25, the modified model of Atsmon-Peake teaches all the limitations of claim 16, and Atsmon also teaches wherein the first digital representation is ascertained from at least one simulation of a driving situation and (Page 24, line 6 -9, the simulated driving data comprises simulated environment data and simulated agent data. Optionally, the simulated environment data comprises a plurality of simulated environment values of the plurality of environment attributes describing the simulated driving environment) the second digital representation is ascertained from at least one recording of measured data recorded using at least one sensor during a drive of a vehicle (Page 8, line 19 – 23, With reference to the first and second aspects, in a seventh possible implementation of the first and second aspects of the present invention the real input data comprises real data collected by at least one sensor selected from a group of sensors comprising: a camera, an electromagnetic radiation sensor, a radar, a Light Detection and Ranging (LIDAR) sensor, a microphone, a thermometer, an accelerometer, and a video camera).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1).
As of claim 17, the modified model of Atsmon-Peake teaches all the limitations of claim 16, but it does not explicitly teach wherein it is binarily detected respectively in one bit in each case whether a subregion is occupied by at least one traffic-relevant object.
While Lang teaches wherein it is binarily detected respectively in one bit in each case whether a subregion is occupied by at least one traffic-relevant object ([0034], An occupancy value of the cell may be determined on the basis of the occupancy probability. The occupancy value may preferably assume a binary value having the values “occupied” or 1 and “free” or 0. The occupancy value may also be a ternary value, which, additionally, may assume a value “unknown,” and, for example, be represented by value ½. In accordance with other specific embodiments, the occupancy value may assume continuous values of between 0 and 1)
Lang is considered to be analogous to the claimed invention since it teaches detecting obstacles in a driving environment of a vehicle. Therefore it would be obvious for a person of ordinary skill in the art before the effective filing data, to integrate Lang’s teaching of assigning a binary number for the driving environment as values “occupied” or 1 and “free” or 0, to compare the two digital representation of driving environment as the modified model teaches.
The motivation would have been improving vehicle safety by detecting obstacles in a driving environment of a vehicle using a radar sensor system and by representing the driving environment of the vehicle as a typically two-dimensional grid structure, each cell of the grid structure being assigned an occupancy value (Lang, [0001] – [0005]).
Claim 18 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1) further in the view of He; Fangning (US10809073B2).
As of claim 18, the modified model of Atsmon-Peake-Lang teaches all the limitations of claim 17, but it does not explicitly teach wherein the bits acquired for all subregions are combined in a binary number as a fingerprint.
While He teaches wherein the bits acquired for all subregions are combined in a binary number as a fingerprint (Col. 29 line 35- 38, The method of any of embodiments 1 to 9, further comprising encoding the 2D occupancy grid as a 2D binary occupancy image, wherein a grid cell contains two possible values).
He is considered to be analogous to the claimed invention since it teaches detected objects or features to a map using a localization process for autonomous vehicles. Therefore it would be obvious for a person of ordinary skill in the art before the effective filing date to integrate He’s teaching of combining the bits of the subregions to create a fingerprint (2D occupancy image) into the modified model to compare two digital representations of a vehicles.
The motivation would have been to providing more accurate 2D occupancy grids along an entire region of a roadway and create a simpler and more efficient map building system. Improved maps allow autonomous vehicles to move safely and reliably in ever-changing environments, providing precise and constantly updated information about their surroundings (He, Col. 4 line 32- 37).
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1) further in the view of He; Fangning (US10809073B2) further in the view of Pink; Oliver (US9823661B2).
As of claim 19, the modified model of Atsmon-Peake-Lang-He teaches all the limitations of claim 18, but it does not explicitly teach wherein a significance of a bit in the binary number is lower or higher the greater the importance of an object in the corresponding subregion for a behavior strategy of the vehicle in the first or second driving situation.
While Pink teaches wherein a significance of a bit in the binary number is lower or higher the greater the importance of an object in the corresponding subregion for a behavior strategy of the vehicle in the first or second driving situation (Col 3-4 line 65-67 , 1-3 Provision can be made for this purpose, for example, that a larger number of cells Z of occupancy grid map 100 are located in regions in which highly accurate environment sensing is necessary for a given driving situation, while fewer cells Z are used in so-called “uninteresting” regions).
Pink is considered to be analogous to the claimed invention since it teaches occupancy grid map for a vehicle as a function of driving situation. Therefore it would be obvious for a person of ordinary skill in the art before the effective filling date to integrate Pink’s teaching of assigning a larger number to the occupancy grid map for an objects which are important in the driving environments into the modified model to compare two digital representations of a vehicles.
The motivation would have been to generate improved occupancy grid map for a vehicle and it helps to increase safety of vehicle since vehicle's behavior is planned on the basis of that description, with the result that the vehicle, for example, can react appropriately to other traffic participants. The environment can be represented in a variety of coordinate systems; a representation in Cartesian coordinates or polar coordinates is known (Pink, Col1. Line 23-32).
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1) further in the view of He; Fangning (US10809073B2) further in the view of Pink; Oliver (US9823661B2) further in the view of Korte; Theodore H. (US11025972B2).
As of claim 20, the modified model of Atsmon-Peake-Lang-He teaches all the limitations of claim 18, and He also teaches binary value for occupancy grids but they does not explicitly teach the similarity measure is a function of a length of a bit sequence that is identical in both the first and second fingerprints.
While Korte teaches the similarity measure is a function of a length of a bit sequence that is identical in both the first and second fingerprints (Col. 12 line 33-35, Hamming distance between two binary vectors of equal length is the number of positions at which the corresponding bits are different). While since the hamming distance is computed, the length of a bit that is identical can be found by subtracting hamming distance from length of the binary.
Korte is considered to be analogous to the claimed invention since it teaches comparing of two fingerprints using Hamming distance. Therefore it would be obvious for a person of ordinary skill in the art before the effective filing date to use Korte’s teaching of Hamming distance similarity measurement into the modified model to compare the first and second fingerprint by computing the length of a bit sequence that is identical on both fingerprints by subtracting hamming distance from the length of binary number.
The motivation would have been by using a similarity measurement like hamming distance to compare two fingerprints it improves the accuracy of finding the matching segments (Col. 18 line 37- 41).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1) in the view of Lang Stefan (US 20200233061 A1) further in the view of Korte; Theodore H. (US11025972B2)
As of claim 21, the modified model Atsmon-Peake-Lang teaches all the limitations of claim 17, but it doesn’t explicitly teach wherein the similarity measure is a function of a Hamming distance between the first and second fingerprints.
While Korte teaches , wherein the similarity measure is a function of a Hamming distance between the first and second fingerprints (Col. 12, line 29- 36, A Hamming distance is calculated between the two fingerprints (i.e., one fingerprint generated from the signal as received at the local probe and the other fingerprint from a signal as received at a remote probe). The Hamming distance between two binary vectors of equal length is the number of positions at which the corresponding bits are different. If two fingerprints are identical the Hamming distance is 0).
Korte is considered to be analogous to the claimed invention since it teaches comparing of two fingerprints using Hamming distance. Therefore it would be obvious for a person of ordinary skill in the art before the effective filing date to use Korte’s teaching of Hamming distance similarity measurement into the modified model to compare the first and second fingerprint.
The motivation would have been by using a similarity measurement like hamming distance to compare two fingerprints it improves the accuracy of finding the matching segments (Col. 18 line 37- 41).
Claims 22 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1)further in the view of Chavali; Pothuraju (US 20190228237 A1).
As of claim 22, the modified model of Atsmon-Peake teaches all the limitations of claim 16, and Atsmon also teaches forming of a first and second fingerprints but it does mot explicitly teach , wherein the occupancies acquired for all subregions are combined in a matrix in each case.
While Chavali teaches wherein the occupancies acquired for all subregions are combined in a matrix in each case ([0038] For example, the occupancy grid may be a two-dimensional matrix where each matrix element represents a probability value for an occupancy at the respective position).
Chavali is considered to be analogous to the claimed invention since it relates to advanced driver assistance system, an autonomous driving vehicle and an industrial robot system. Therefore it would be obvious to try for a person of ordinary skill in the art before the effective filing date to integrate Chavali’s teaching of creating a 2D matrix for the occupancy grid where each matrix element represents a probability value for an occupancy at the respective position into the modified model of the first and second fingerprints.
The motivation would have to improve advanced driver assistance system, an autonomous driving vehicle and an industrial robot system by been efficient and reliable determination of a boundary between the free space and the occupied space in the environment of a vehicle by detecting free space in the surrounding of a vehicle (Chavali, [0002- 0005]).
As of claim 24, the modified model Atsmon-Peake- Chavali teaches all the limitations of claim 22, and Atsmon also teaches wherein the similarity measure includes an average value or median of elements in an elementwise product of both the first and second fingerprints (page 9, line 15 -21, computing the model score comprises computing at least one term selected from a group of terms consisting of: an average of the plurality of image self-distance scores, a standard deviation of the plurality of image self-distance scores, a sum of the plurality of image self-distance scores, an average of the plurality of realism scores, a standard deviation of the plurality of realism scores, a sum of the plurality of realism scores, an average of the plurality of image quality scores, a standard deviation of the plurality of image quality scores, and a sum of the plurality of image quality scores)
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1)further in the view of Chavali; Pothuraju (US 20190228237 A1) further in the view of Haitsma; Jaap Andre (US 7549052 B2).
As of claim 23, the modified model of Atsmon-Peake- Chavali teaches all the limitations of claim 22, but it does not explicitly teach wherein at least the matrix generated from one of the first or second digital representations is converted into the first or second fingerprint with the aid of filtering using at least one filter core.
While Haitsma teaches wherein at least the matrix generated from one of the first or second digital representations is converted into the first or second fingerprint with the aid of filtering using at least one filter core (the robust audio hashes are derived from an audio signal by comparing energy in different frequency bands and over time. A generalization of this approach is to consider any cascade of LTI and non-linear functions. In particular, a robust hash can also be obtained by applying a (dyadic) filter bank (an LTI operator), followed by squaring or taking absolute words (a non-linear function), followed by a difference operator over time and/or band (an LTI operator), finally followed by a thresholding operator. By applying a carefully designed linear filter bank as an initial operator, the complexity of a FFT can be avoided).
Haitma is considered to be analogous to the claimed invention since it teaches generating and matching hashes of multimedia content. Therefore it would be obvious to try for a person of ordinary skill in the art before the effective filing date to apply filter on the matrix convert it into a fingerprint of the modified model based on Haitsma’s teaching of applying filter on the content to generate a robust hash.
The motivation would have been by using matching strategy and applying filter on the hash the extraction process also provides information (19) as to which of the hash bits are the least reliable and flipping these bits considerably improves the speed and performance of the matching process (Haitma, abstract).
Claims 26 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over ATSMON DAN (WO 2020079685 A1) in the view of Peake; Miguel Alexander (US20200074266A1)further in the view of Dolan; James Graham (US 11415997 B1).
As of claim 26, Atsmon teaches A method for validating a simulation of a drive of a vehicle, the method comprising the following steps (Page 1 line 1-15, When generating simulated data for training an autonomous driving system there is a need to generate data that exhibits realistic behavior and a realistic look (for example for visual sensors).
comparing the first digital representation of the simulated driving situation against each of the second digital representations of the reference driving situations, (page 13 line 6- 11, In computing, a statistical fingerprint of identified data is a statistical value computed using the identified data that uniquely identifies the identified data for practical comparison purposes. The present invention additionally proposes in some embodiments thereof classifying the realism of the simulated driving environment using a difference between a simulation statistical fingerprint computed using simulated driving data collected from a simulated driving, and a real statistical fingerprint computed using the real traffic data) the first and second digital representations including occupancy information about an occupancy of the environment of the vehicle by traffic-relevant objects, the comparing including, for each of the second digital representations (Page 12, line 24- 31, Instead of randomly generating driving characteristics of a simulated agent, or randomly generating a simulated agent’ s behavior at every point of a simulation, the present invention, in some embodiments thereof, proposes learning realistic moving object characteristics and realistic movement patterns from real traffic data collected from real traffic environments. In such embodiments the present invention further proposes generating one or more simulated agents according to the learned moving object characteristics and realistic movement patterns, such that each simulated agent embodies a realistic type of moving object (that is, vehicle or pedestrian) and a realistic type of movement pattern in a realistic traffic environment).
combining them to form a first fingerprint of the first driving situation; (page 6, line 13 -14, The real statistical fingerprint is computed using the plurality of environment values and the plurality of agent values)
combining them to form a second fingerprint of the second driving situation; (page 6, line 21- 23, the simulation statistical fingerprint is computed using the plurality of simulation environment attribute values and the plurality of simulation agent values).
ascertaining a similarity measure between the first fingerprint and the second fingerprint according to a predefined measure specification; and (Page 8, line 13 – 16, modifying a plurality of other model parameters of the other simulation generation model to minimize another difference between another simulation statistical fingerprint, computed using the other simulated driving data, and the real statistical fingerprint, computed using the real input data…Page 13 line 8- 11, classifying the realism of the simulated driving environment using a difference between a simulation statistical fingerprint computed using simulated driving data collected from a simulated driving, and a real statistical fingerprint computed using the real traffic data).
determining that the first and second driving situations are identical or at least similar based on the similarity measure satisfying a predefined criterion (Page 13 line 6- 11, In computing, a statistical fingerprint of identified data is a statistical value computed using the identified data that uniquely identifies the identified data for practical comparison purposes. The present invention additionally proposes in some embodiments thereof classifying the realism of the simulated driving environment using a difference between a simulation statistical fingerprint computed using simulated driving data collected from a simulated driving, and a real statistical fingerprint computed using the real traffic data… Page 27 line 5 – 8, A realism score of a refiner measures how realistic a refined image generated by the refiner in response to an input image appears. In 812, processing unit optionally computes a plurality of realism scores, each indicative of a quality of realism of one of the plurality of refined output images. Optionally each of the plurality of realism scores is normalized in a range from 0 to 1). Realism is used as a predefined criterion to check its similarity.
While Atsmon does not explicitly teach converting at least one simulated driving situation of a simulated drive into a first digital representation, ascertaining second digital representations of a multitude of reference drives within the reference drives from time characteristics of reference drives; subdividing a region of the environment of the vehicle into a grid of subregions: ascertaining, based on the occupancy information in the first digital representation, occupancies of the subregions by traffic-relevant objects, ascertaining, based on the occupancy information in the second digital representation, occupancies of the subregions by traffic-relevant objects, ascertaining from results of the comparisons at least one reference driving situation within a reference drive that is identical or at least similar to the simulated driving situation; comparing a characteristic of the simulated drive following the simulated drive situation with a characteristic of the reference drive following the ascertained reference driving situation; and determining that the simulation of the drive is valid at least for the simulated driving situation when the following characteristic of the simulated drive is in line according to the predefined criterion with the following characteristic of the reference drive.
While Peake teaches converting at least one simulated driving situation of a simulated drive into a first digital representation; ([0096], In some embodiments, a scenario simulator (not shown) may be configured to generate one or more simulated environment scenarios, wherein each of the simulated environment scenario(s) corresponds to a variation of a particular object, surface, or situation within the virtual environment), the simulated driving data is considered as the first digital representation… [0119], In some embodiments, the perception signals 508 include data representing “occupancy grids” (e.g., one grid per T milliseconds), with each occupancy grid indicating object positions (and possibly object boundaries, orientations, etc.) within an overhead view of the autonomous vehicle's environment. Within the occupancy grid, each “cell” (e.g., pixel) may be associated with a particular class as determined by the classification module 512, possibly with an “unknown” class for certain pixels that were not successfully classified).
subdividing a region of the environment of the vehicle into a grid of subregions; (Abstract, an occupancy grid generator is used to generate an occupancy grid indicative of an environment of an autonomous vehicle from an imaging scene that depicts the environment…[0019] In some embodiments, the perception signals 508 include data representing “occupancy grids” (e.g., one grid per T milliseconds), with each occupancy grid indicating object positions (and possibly object boundaries, orientations, etc.) within an overhead view of the autonomous vehicle's environment).
ascertaining, based on the occupancy information in the first digital representation, occupancies of the subregions by traffic-relevant objects, and( [0119], In some embodiments, the perception signals 508 include data representing “occupancy grids” (e.g., one grid per T milliseconds), with each occupancy grid indicating object positions (and possibly object boundaries, orientations, etc.) within an overhead view of the autonomous vehicle's environment. Within the occupancy grid, each “cell” (e.g., pixel) may be associated with a particular class as determined by the classification module 512, possibly with an “unknown” class for certain pixels that were not successfully classified).based on the occupancy grids it shows the traffic relevant object which is included in perception signal so this is interpreted as the first digital representation.
ascertaining, based on the occupancy information in the second digital representation, occupancies of the subregions by traffic-relevant objects, and ([0119], Similarly, the prediction signals 522 may include, for each such grid generated by the perception component 506, one or more “future occupancy grids” that indicate predicted object positions, boundaries and/or orientations at one or more future times (e.g., one, two, and five seconds ahead). Occupancy grids are discussed further below in connection with FIGS. 6A and 6B). perception signal 522 including the future occupancy is considered as the second digital representation.
Atsmon and Peake is considered as analogous to the claimed invention since they focus on training vehicles using simulation realist data. Therefore it would be obvious for a person of ordinary skill in the art, before the effective filling date to determine a similarity using a fingerprint of an occupancy map and set a similarity threshold by combining Atsmon’s teaches of actual and simulated fingerprint to determine the similarity between a real environment and a simulated driving environment, and Peake’s teaching of occupancy grid and determining traffic relevant objects in subregions of the environment.
The motivation would have been to improve the efficiency and effectiveness of generating and/or collecting numerous autonomous driving datasets, and also address safety concerns with respect to generating sufficient datasets in a non-dangerous and controlled manner when training autonomous vehicles in real-world driving applications (Peake, [0006]).
The modified model does not explicitly teach, ascertaining second digital representations of a multitude of reference drives within the reference drives from time characteristics of reference drives; ascertaining from results of the comparisons at least one reference driving situation within a reference drive that is identical or at least similar to the simulated driving situation; comparing a characteristic of the simulated drive following the simulated drive situation with a characteristic of the reference drive following the ascertained reference driving situation; and determining that the simulation of the drive is valid at least for the simulated driving situation when the following characteristic of the simulated drive is in line according to the predefined criterion with the following characteristic of the reference drive.
While Dolan teaches ascertaining second digital representations of a multitude of reference drives within the reference drives from time characteristics of reference drives; (Col. 8 line 23 -28, In some instances, the log data captured by vehicle 112 and simulated vehicle 116 may include time and position data identifying the location of the vehicles 112, 116 within their respective environments… Col. 10 line 45- 50, In some cases, the simulation system may analyze multiple previous log data (e.g., based on the distances between the vehicle in the log data and the simulated vehicle), to determine which of the previous log data is optimal for a log-based simulation using the simulated vehicle), the second representation also determined by comparing and analyzing the previous log based simulation. The log data is also a based-on time characteristics. Since it includes time and position.
ascertaining from results of the comparisons at least one reference driving situation within a reference drive that is identical or at least similar to the simulated driving situation; (Col.4 line 20 – 27, The simulated scenario can be identical to the captured environment or deviate from the captured environment. For example, the simulated environment can have more or fewer number of buildings as in the captured environment. Additionally, objects (e.g., other vehicles, pedestrians, animals, cyclists, etc.) in the log data can be represented as simulated objects and the vehicle can be represented as a simulated vehicle…Col. 10 line 34- 37, In operation 110, before generating new log data, the simulation system may analyze each of the previously stored log data to determine if any of the previous log data is valid for the log-based simulation.). This shows the system is searching for a reference data from the previously stored data to select a reference and the simulated scenario can be identical to the captured/ real environment which is used as a reference.
comparing a characteristic of the simulated drive following the simulated drive situation with a characteristic of the reference drive following the ascertained reference driving situation; and (Col. 8 line 29- 37, the relative locations of the vehicles 112, 116 also may be computed in some cases based on additional sensor data, such as the distances, angles, and detection times of common objects identified within the log data and the corresponding log-based simulation. The driving simulation system may calculate and monitor the distance between vehicles 112 and 116 during the log-based simulation, and may compare the distance to a predetermined threshold to determine whether the log-based simulation is valid or invalid) as it is shown above the simulated drive is compared with the log- based simulation which is a real data and used as a reference drive).
determining that the simulation of the drive is valid at least for the simulated driving situation when the following characteristic of the simulated drive is in line according to the predefined criterion with the following characteristic of the reference drive (Col. 7-8, line 61- 67, 1 – 5, In some examples, the driving simulation system may determine that a log-based simulation is invalid for the vehicle control system simulating the vehicle, based on responses received to the simulation from the simulation system or from the vehicle control system of the simulated vehicle 116. For instance, the control system of the simulated vehicle 116 may analyze the simulated environment and/or objects during the simulation to determine object consistency, jerkiness, driving patterns and speeds, etc., and may compare one or more of these factors to threshold(s) to determine whether the simulation is valid or invalid).
Dolan is considered to be analogous to the claimed invention since it teaches autonomous driving simulation using simulated and real data. Therefore it would be obvious for a person of ordinary skill in the art before the effective filing date to integrate Dolan ‘s teaching of validating a driving simulation against a real world behaviors by comparing the simulated vehicle with a threshold into the modified model to validate the simulation by comparing two digital representations of a statistical fingerprint of real world environment variables and simulated environment variable to determine similarity between a real and simulated driving environment.
The motivation would have been improving the operation of simulation systems and the quality and efficacy of driving simulations. For example, by determining that a log-based simulation is invalid and then performing a new virtual driving simulation to generate new log data, greater numbers of resource-efficient log-based tests may be performed for a simulation scenario. Additionally, the log-based simulation techniques described herein may be performed sequentially on different components of the simulated vehicle, which may require less memory and computational resources than virtual simulations requiring all of the components of the simulated vehicle and additional simulation objects to be instantiated and executed at the same time (Dolan, Col6 line 1- 11).
As of claim 27, the modified model of Atsmon-Peake- Dolan teaches all the limitations of claim 26, and Dolan also teaches wherein brake danger parameters and/or required steering potentials reached in the following characteristics of the simulated and the reference drive to mitigate the danger are used for the comparison of the following characteristics (Col. 17 -18, line 63- 67 , 1-5, The drive system(s) 714 can include many of the vehicle systems, including a high voltage battery, …a steering system including a steering motor and steering rack (which can be electric), a braking system including hydraulic or electric actuators, a suspension system including hydraulic and/or pneumatic components, a stability control system for distributing brake forces to mitigate loss of traction … Col. 20 line 37- 40, a planning component 728 may determine there is no such collision free path and, in turn, provide a path which brings the vehicle to a safe stop avoiding all collisions and/or otherwise mitigating damage …Col. 24 line 7 – 19, the simulation component 748 generate the simulation data indicating how the vehicle control system 702 performed (e.g., responded) and can compare the simulation data to a predetermined outcome and/or determine if any predetermined rules/assertions were broken/triggered… In some instances, the predetermined rules/assertions can be based on the simulated scenario (e.g., traffic rules regarding crosswalks can be enabled based on a crosswalk scenario or traffic rules regarding crossing a lane marker can be disabled for a stalled vehicle scenario). As it is cited , Dolan teaches how safe the control system response to the rules /assertions and it includes a braking , steering system and a stability control system for distributing brake forces to mitigate loss of traction, so it would be obvious to use a braking/steering parameter to mitigate danger in the a stability control system.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
BUERKLE; Cornelius (US 20200326721 A1, Date Published, 2020-10-15) is similar to the claimed invention since it teaches occupancy grid data representing a plurality of cells of an occupancy grid and for each cell of the plurality of cells, a probability of whether the cell is occupied; receive object location data, representing one or more locations of one or more objects identified using a model and compare the occupancy grid data to the comparison grid data using at least one comparing rule.
Alvarez; Ignacio (US 20190050520 A1, Date Published 2019-02-14) is similar to the claimed invention since it teaches a simulated vehicle modeling system is adapted to obtain a vehicle performance fingerprint, such as from a vehicle performance fingerprint that includes vehicle performance data collected from a unique vehicle while experiencing real world driving condition.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABRHAM A. TAMIRU whose telephone number is (571)272-6987. The examiner can normally be reached Monday - Friday 8:00am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ryan Pitaro can be reached at 571 272 4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABRHAM ALEHEGN TAMIRU/ Examiner, Art Unit 2188
/RYAN F PITARO/ Supervisory Patent Examiner, Art Unit 2188
Prosecution Insights