Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to because they do not provide enough information to be useful without also reading the specification, specifically, Figure 5. Figure 5 shows a flow chart, but the boxes are blank, other than pointing towards the specification with labels such as S1 or S2. This figure effectively provides no information. If the steps were described with text in some way in the flow chart so that it would convey information alone, it would be sufficient to overcome the objection. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “central processing device” and “assessment device” in claims 1-19.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-14 and 19 are rejected under 35 U.S.C. 101 because they are directed towards a mental process without significantly more.
Claim 1 cites:
A safety system assembly for monitoring a zone in which objects move together,
wherein the safety system assembly comprises a central processing device that is configured to receive sensor data from a plurality of monitoring units, in which sensor data the objects detected in the monitored zone by the monitoring units are included,
wherein the central processing device is configured to consolidate the received sensor data,
wherein the central processing device is configured to create object lists from the consolidated sensor data, with the object lists including the detected objects together with the respective object information, and to transmit these object lists to the autonomously driving vehicles.
Step 2A prong one evaluation: Judicial Exception – Yes – Mental Processes
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under its broadest reasonable interpretation, the claim covers performance using mental.
The claims recite consolidating received sensor data. This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could take data and mentally consider it as a whole, thus consolidating the data. Thus this step is directed to a mental process.
The claims recite creating object lists . This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could consider data, and create object lists from observed objects. Thus this step is directed to a mental process.
Step 2A Prong Two evaluations
Claims are evaluated whether as a whole it integrates the recited judicial exception into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea or adding/performing insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”).
The claims recite consolidating data and creating object lists using a device, central processing device, or an AI module. The above listed actions are recited at a high level of generality. The computer/circuitry that facilitate the steps are described by the specification at a high level of generality. The generically recited computer merely describes how to generally “apply” the otherwise mental/extra solution processes using a generic or general-purpose processor. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claims recite receiving and transmitting sensor data. The previously listed action is described at a high level of generality. The sending, receiving and production of signals is considered well known, common, and conventional. Producing signals, sending and receiving data and performing functions known in the art is considered insignificant extra solution activity. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claim is not patent eligible.
2B Evaluation: Inventive Concept – No
Claims are evaluated as to whether the claims as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
As discussed with respect to Step 2A Prong Two, the additional elements in the claim amount to no more than possible uses for the output of the abstract idea. The same analysis applies here in 2B, i.e., possible uses for information or mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus the claims are not patent eligible.
Claim 19 cites:
A method of monitoring a zone, in which objects move together, comprising the following method steps:
- receiving sensor data from a plurality of monitoring units, in which sensor data the objects detected in the monitored zone by the monitoring units are included;
- consolidating the received sensor data;
- creating object lists from the consolidated sensor data, wherein the object lists include the detected objects together with the respective object information;
- transmitting these object lists to the autonomously driving vehicles.
Step 2A prong one evaluation: Judicial Exception – Yes – Mental Processes
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under its broadest reasonable interpretation, the claim covers performance using mental.
The claims recite consolidating received sensor data. This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could take data and mentally consider it as a whole, thus consolidating the data. Thus this step is directed to a mental process.
The claims recite creating object lists . This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could consider data, and create object lists from observed objects. Thus this step is directed to a mental process.
Step 2A Prong Two evaluations
Claims are evaluated whether as a whole it integrates the recited judicial exception into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea or adding/performing insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”).
The claims recite consolidating data and creating object lists using a device, central processing device, or an AI module. The above listed actions are recited at a high level of generality. The computer/circuitry that facilitate the steps are described by the specification at a high level of generality. The generically recited computer merely describes how to generally “apply” the otherwise mental/extra solution processes using a generic or general-purpose processor. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claims recite receiving and transmitting sensor data. The previously listed action is described at a high level of generality. The sending, receiving and production of signals is considered well known, common, and conventional. Producing signals, sending and receiving data and performing functions known in the art is considered insignificant extra solution activity. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claim is not patent eligible.
2B Evaluation: Inventive Concept – No
Claims are evaluated as to whether the claims as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
As discussed with respect to Step 2A Prong Two, the additional elements in the claim amount to no more than possible uses for the output of the abstract idea. The same analysis applies here in 2B, i.e., possible uses for information or mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus the claims are not patent eligible.
Claim 2 cites:
The safety system assembly according to claim 1, wherein the zone is one of a warehouse and a factory.
Claim 3 cites:
The safety system assembly according to claim 1, wherein the objects are at least one of autonomously driving vehicles and persons.
Claim 4 cites:
The safety system assembly according to claim 1
wherein the object information of an object comprises a position, size, direction of movement and/or speed of movement, object ID and/or object class for the object.
Claim 5 cites:
The safety system assembly according to claim 1
wherein the central processing device is configured to convert the sensor data of the plurality of monitoring units into a common spatial and temporal coordinate system.
Claim 6 cites:
The safety system assembly according to claim 1, wherein the central processing device is configured to subject the received sensor data to a plausibility check by checking whether:
a) a respective object is included in the sensor data from at least two monitoring units whose monitoring zones at least partly overlap; and/or
b) a respective moving object is included, in different but mutually adjoining time periods, in the sensor data from at least two monitoring units whose monitoring zones adjoin one another.
Claim 7 cites:
The safety system assembly according to claim 1
wherein the safety system assembly comprises an assessment device that is configured to determine a confidence level for an object based on the sensor data and/or the object information.
Claim 8 cites:
The safety system assembly according to claim 7
wherein the assessment device is configured to define a higher confidence level for an object if the object is included in sensor data from at least two monitoring units that were produced at the same time and whose monitoring fields at least partly overlap.
Claim 9 cites:
The safety system assembly according to claim 7
wherein the assessment device is configured to determine the confidence level for the object based on the quality of the sensor data and/or of the object information.
Claim 10 cites:
The safety system assembly according to claim 9
wherein the quality of the: a) sensor data depends on physical properties of the respective monitoring unit;
and/or b) object information depends on the position, direction of movement and/or object class.
Claim 11 cites:
The safety system assembly according to claim 10, wherein the physical properties comprise the age, the type, the error rate, the failure rate, the scatter rate, the measurement method, the installation location and/or confidence information of the respective monitoring unit.
Claim 12 cites:
The safety system assembly according to claim 10, wherein the object information depends on whether it is a person or an autonomously driving vehicle.
Claim 13 cites:
The safety system assembly according to claim 7
wherein the assessment device comprises an Al module and wherein the Al module is configured to determine the confidence level for an object based on the sensor data and/or the object information.
Claim 14 cites:
The safety system assembly according to claim 7
wherein the safety system assembly comprises at least one autonomously driving vehicle, wherein the at least one autonomously driving vehicle is configured to receive the object list from the central processing device.
Step 2A prong one evaluation: Judicial Exception – Yes – Mental Processes
The Office submits that the foregoing bolded limitation(s) constitutes judicial exceptions in terms of “mental processes” because under its broadest reasonable interpretation, the claim covers performance using mental.
Claim 5 recites converting sensor data to a common spatial and temporal coordinate system. This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could take sensor data and convert it to a common spatial and temporal coordinate system in their mind, or using a pen and paper. Thus this step is directed to a mental process.
Claim 6 recites subjecting received sensor data a plausibility check. This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could consider if the object was detected by two overlapping sensors, and determine if the data is plausible. Thus this step is directed to a mental process.
Claim 7 recites determining a confidence level for an object based on sensor data and object information. This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could consider sensor data or object data and determine if they are confident in the data and assign a value. Thus this step is directed to a mental process.
Claim 8 recites defining a higher confidence level if overlapping sensors detect an object at the same time. This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could that the sensors overlap when seeing the object, and increase the confidence level based on that. Thus this step is directed to a mental process.
Claim 9 recites determining the confidence level based on the quality of sensor data and object information. This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could consider the sensor data or object information, and determine how confident they are that the data is accurate based on that. Thus this step is directed to a mental process.
Claim 13 recites determining a confidence level based on sensor data or object information. This limitation, as drafted, is a simple process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind, but for the limitation that processing circuitry be programed to perform the task. That is, other than reciting “processor”, or “memory”, nothing in the claim precludes the element being done in the mind. A person could consider the sensor data or object information, and determine how confident they are that the data is accurate based on that. Thus this step is directed to a mental process.
Step 2A Prong Two evaluations
Claims are evaluated whether as a whole it integrates the recited judicial exception into a practical application. As noted in the 2019 PEG, it must be determined whether any additional elements in the claim beyond the abstract idea integrate the exception into a practical application in a manner that imposes a meaningful limit on the judicial exception. The courts have indicated that additional elements merely using a computer to implement an abstract idea or adding/performing insignificant extra solution activity, or generally linking use of a judicial exception to a particular technological environment or field of use do not integrate a judicial exception into a “practical application.”
In the present case, the additional limitations beyond the above-noted abstract idea are as follows (where the underlined portions are the “additional limitations” while the bolded portions continue to represent the “abstract idea”).
The claims recite consolidating data, converting data to coordinate systems, subjecting data to plausibility checks, determining confidence levels, and creating object lists using a device, central processing device, and assessment device, or an AI module. The above listed actions are recited at a high level of generality. The computer/circuitry that facilitate the steps are described by the specification at a high level of generality. The generically recited computer merely describes how to generally “apply” the otherwise mental/extra solution processes using a generic or general-purpose processor. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claims recite receiving and transmitting sensor data. The previously listed action is described at a high level of generality. The sending, receiving and production of signals is considered well known, common, and conventional. Producing signals, sending and receiving data and performing functions known in the art is considered insignificant extra solution activity. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
The claim is not patent eligible.
2B Evaluation: Inventive Concept – No
Claims are evaluated as to whether the claims as a whole amount to significantly more than the recited exception, i.e., whether any additional element, or combination of additional elements, adds an inventive concept to the claim.
As discussed with respect to Step 2A Prong Two, the additional elements in the claim amount to no more than possible uses for the output of the abstract idea. The same analysis applies here in 2B, i.e., possible uses for information or mere instructions to apply an exception on a generic computer cannot integrate a judicial exception into a practical application at Step 2A or provide an inventive concept in Step 2B.
Thus the claims are not patent eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-14, and 18-19 are rejected under 35 U.S.C. 103 as being unpatentable over Taylor et al (US Pub 2019/0333785 A1) in light of Moustafa et al (US Pub 2022/0126864 A1), hereafter known as Moustafa.
For Claim 1, Taylor teaches A safety system assembly for monitoring a zone in which objects move together,
wherein the safety system assembly comprises a central processing device that is configured to receive sensor data from a plurality of monitoring units, in which sensor data the objects detected in the monitored zone by the monitoring units are included, ([0112] In some embodiments, the map may be determined based on data from sensor 806, other sensors on other vehicles within the environment, or other sensors positioned at fixed locations within the environment. The map may represent a fusion of information from multiple different sensors, or from multiple different types of sensors. In some examples, the map may be a two-dimensional representation of the environment, and may be determined or updated by projecting representations of physical features within the environment detected by the sensors onto a plane defined by the two-dimensional representation of the environment. The map may represent a threshold extent of the environment around the vehicle (e.g., a 10 meter by 10 meter square centered around the vehicle). Alternatively, the map may represent the environment in three dimensions, and may therefore represent a volume of space centered around the vehicle.)
wherein the central processing device is configured to consolidate the received sensor data, ([0112] In some embodiments, the map may be determined based on data from sensor 806, other sensors on other vehicles within the environment, or other sensors positioned at fixed locations within the environment. The map may represent a fusion of information from multiple different sensors, or from multiple different types of sensors. In some examples, the map may be a two-dimensional representation of the environment, and may be determined or updated by projecting representations of physical features within the environment detected by the sensors onto a plane defined by the two-dimensional representation of the environment. The map may represent a threshold extent of the environment around the vehicle (e.g., a 10 meter by 10 meter square centered around the vehicle). Alternatively, the map may represent the environment in three dimensions, and may therefore represent a volume of space centered around the vehicle.)
Taylor does not explicitly teach wherein the central processing device is configured to create object lists from the consolidated sensor data, with the object lists including the detected objects together with the respective object information, and to transmit these object lists to the autonomously driving vehicles.
Moustafa, however, does teach wherein the central processing device is configured to create object lists from the consolidated sensor data, with the object lists including the detected objects together with the respective object information, and to transmit these object lists to the autonomously driving vehicles. ([0226] Turning to FIG. 10, a simplified block diagram 1000 is shown illustrating an enhanced autonomous driving system including blocks (e.g., 1005, 1035, 1040, 244, etc.) providing functionality to intelligently manage the creation, storage, and offloading of sensor data generated by a sensor array 1005 on a corresponding autonomous vehicle. For instance, the sensor array 1005 may be composed of multiple different types of sensors (such as described herein) and may be further provided with pre-processing software and/or hardware to perform some object recognition and provide object list results as well as raw data. In some implementations, the pre-processing logic may also assist in optimizing data delivery and production. Data from the sensor array 1005 may be provided to an in-vehicle data reservoir 1010 (or memory), which may be accessed and used by other functional blocks of the autonomous driving system. For instance, an autonomous driving stack 1015 using various artificial intelligence logic and machine learning models may receive or retrieve the sensor data to generate outputs to the actuation and control block 1020 to autonomously steer, accelerate, and brake the vehicle 105. In some cases, results generated by the autonomous driving stack 1015 may be shared with other devices (e.g., 1025) extraneous to the vehicle 105.)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa so that object lists with information are created and sent to the autonomous vehicle, because it would allow the autonomous vehicle to take corrective action to avoid colliding with the obstacles, especially if there are control elements and controllers making decisions on the autonomous vehicle itself.
For Claim 2, Taylor teaches The safety system assembly according to claim 1, wherein the zone is one of a warehouse and a factory. ([0034] Robotic navigation in an environment such as a warehouse can allow autonomous robotic devices the ability to carry out one or more actions without the need for constant human control. When a robotic device is given a task, a corresponding path through the environment and trajectory for traversing the path may be determined for the task. The path may include an ordered sequence of locations or positions for the robotic device within the environment. The trajectory may include timing, speed, and orientation of the robotic device for each of the positions along the path.)
For Claim 3, Taylor teaches The safety system assembly according to claim 1, wherein the objects are at least one of autonomously driving vehicles and persons. ([0145] Further, in some examples, the control system may be configured to classify detected objects into a plurality of categories and determine the first or second threshold distance for each object based on the category into which the object is classified. Classification may be performed using, for example, a machine learning algorithm (e.g., artificial neural networks applied to images from a ToF camera). For example, objects may be generally classified into fixed objects (e.g., walls, pallet racks) and non-fixed objects (e.g., boxes, pallets, vehicles), with fixed objects assigned smaller threshold distances than non-fixed objects. Object may also be classified into more specific categories, such as pallet jack, human, pallet, box, stationary robot, etc., depending on the contents of the environment in which the vehicle is operating.)
For Claim 4, Taylor teaches The safety system assembly according to claim 1,
wherein the object information of an object comprises a position, size, direction of movement and/or speed of movement, object ID and/or object class for the object. ([0145] Further, in some examples, the control system may be configured to classify detected objects into a plurality of categories and determine the first or second threshold distance for each object based on the category into which the object is classified. Classification may be performed using, for example, a machine learning algorithm (e.g., artificial neural networks applied to images from a ToF camera). For example, objects may be generally classified into fixed objects (e.g., walls, pallet racks) and non-fixed objects (e.g., boxes, pallets, vehicles), with fixed objects assigned smaller threshold distances than non-fixed objects. Object may also be classified into more specific categories, such as pallet jack, human, pallet, box, stationary robot, etc., depending on the contents of the environment in which the vehicle is operating.
[0039] A buffer region may be placed around each of the objects identified as an obstacle. The buffer region may operate to enforce a minimum distance (i.e., a first threshold distance) away from the obstacle at which the vehicle is to stop to avoid colliding with the obstacle. The minimum distance may be based on a size of the obstacle, a type or classification of the obstacle, a speed of the vehicle, a size of the vehicle, a load carried by the vehicle, and/or a task assigned to the vehicle, among other factors.
For Claim 5, Taylor teaches The safety system assembly according to claim 1
wherein the central processing device is configured to convert the sensor data of the plurality of monitoring units into a common spatial and temporal coordinate system. ([0113] In some embodiments, the map may be represented as an occupancy grid that includes a number of cells that represent corresponding areas in the environment. The control system may develop the occupancy grid within a coordinate frame that corresponds to a point-of-view (POV) of the sensor (e.g., sensor 806). Each cell may be assigned a state that indicates the status of the area represented by the cell. Particularly, a cell may be assigned as having an obstacle, free space, or unknown. Cells with obstacles may represent physical features within the environment, including fixed, movable, and mobile objects. Cells with free space may be traversable by the vehicle without striking objects in the environment. Unknown cells may require additional sensor data to determine whether the area includes an obstacle or not (i.e., has free space). The control system may periodically update and adjust the occupancy grid based on new measurements of the environment from sensors coupled to one or more vehicles navigating the environment.)
For Claim 6, Taylor teaches The safety system assembly according to claim 1,
Taylor does not explicitly teach wherein the central processing device is configured to subject the received sensor data to a plausibility check by checking whether:
a) a respective object is included in the sensor data from at least two monitoring units whose monitoring zones at least partly overlap; and/or
b) a respective moving object is included, in different but mutually adjoining time periods, in the sensor data from at least two monitoring units whose monitoring zones adjoin one another.
Moustafa, however, does teach wherein the central processing device is configured to subject the received sensor data to a plausibility check by checking whether:
a) a respective object is included in the sensor data from at least two monitoring units whose monitoring zones at least partly overlap; and/or ([0297] FIG. 25 is a simplified diagram showing an example process of rating and validating crowdsourced autonomous vehicle sensor data in accordance with at least one embodiment. In the example shown, each autonomous vehicle 2502 collects data from one or more sensors coupled thereto (e.g., camera(s), LIDAR, radar, etc.). The autonomous vehicles 2502 may use the sensor data to control one or more aspects of the autonomous vehicle. As each autonomous vehicle collects data from its one or more sensors, the autonomous vehicle may determine an amount of confidence placed in datum collected. For example, the confidence score may be based on information related to the collection of the sensor data, such as, for example, weather data at the time of data collection (e.g., camera information on a sunny day may get a larger confidence score than cameras on a foggy day), sensor device configuration information (e.g., a bitrate or resolution of the camera stream), sensor device operation information (e.g., bit error rate for a camera stream), sensor device authentication status information (e.g., whether the sensor device has been previously authenticated by the autonomous vehicle, as described further below), or local sensor corroboration information (e.g., information indicating that each of two or more cameras of the autonomous vehicle detected an object in the same video frame or at the same time).)
b) a respective moving object is included, in different but mutually adjoining time periods, in the sensor data from at least two monitoring units whose monitoring zones adjoin one another.
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa such that a plausibility check in which it is considered if the object was detected by two sensors with overlap because it would prevent the occurrence of false positives. My ensuring that the object was detected twice, it’s location is expected to be more accurate, and the problems caused by malfunctioning sensors can be reduced.
For Claim 7, Taylor teaches The safety system assembly according to claim 1
Taylor does not explicitly teach wherein the safety system assembly comprises an assessment device that is configured to determine a confidence level for an object based on the sensor data and/or the object information.
Moustafa, however, does teach wherein the safety system assembly comprises an assessment device that is configured to determine a confidence level for an object based on the sensor data and/or the object information.
([0297] FIG. 25 is a simplified diagram showing an example process of rating and validating crowdsourced autonomous vehicle sensor data in accordance with at least one embodiment. In the example shown, each autonomous vehicle 2502 collects data from one or more sensors coupled thereto (e.g., camera(s), LIDAR, radar, etc.). The autonomous vehicles 2502 may use the sensor data to control one or more aspects of the autonomous vehicle. As each autonomous vehicle collects data from its one or more sensors, the autonomous vehicle may determine an amount of confidence placed in datum collected. For example, the confidence score may be based on information related to the collection of the sensor data, such as, for example, weather data at the time of data collection (e.g., camera information on a sunny day may get a larger confidence score than cameras on a foggy day), sensor device configuration information (e.g., a bitrate or resolution of the camera stream), sensor device operation information (e.g., bit error rate for a camera stream), sensor device authentication status information (e.g., whether the sensor device has been previously authenticated by the autonomous vehicle, as described further below), or local sensor corroboration information (e.g., information indicating that each of two or more cameras of the autonomous vehicle detected an object in the same video frame or at the same time).)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa such that a confidence level is determined based on sensor or object information because a confidence level for the object would allow the vehicle to respond appropriately. If the vehicle does not know exactly where an object is, it may want to be more cautious. If it knows precisely where it is, it can create a precise trajectory around that exact information.
For Claim 8, Taylor teaches The safety system assembly according to claim 7
Taylor does not explicitly teach wherein the assessment device is configured to define a higher confidence level for an object if the object is included in sensor data from at least two monitoring units that were produced at the same time and whose monitoring fields at least partly overlap.
Moustafa, however, does teach wherein the assessment device is configured to define a higher confidence level for an object if the object is included in sensor data from at least two monitoring units that were produced at the same time and whose monitoring fields at least partly overlap. ([0297] FIG. 25 is a simplified diagram showing an example process of rating and validating crowdsourced autonomous vehicle sensor data in accordance with at least one embodiment. In the example shown, each autonomous vehicle 2502 collects data from one or more sensors coupled thereto (e.g., camera(s), LIDAR, radar, etc.). The autonomous vehicles 2502 may use the sensor data to control one or more aspects of the autonomous vehicle. As each autonomous vehicle collects data from its one or more sensors, the autonomous vehicle may determine an amount of confidence placed in datum collected. For example, the confidence score may be based on information related to the collection of the sensor data, such as, for example, weather data at the time of data collection (e.g., camera information on a sunny day may get a larger confidence score than cameras on a foggy day), sensor device configuration information (e.g., a bitrate or resolution of the camera stream), sensor device operation information (e.g., bit error rate for a camera stream), sensor device authentication status information (e.g., whether the sensor device has been previously authenticated by the autonomous vehicle, as described further below), or local sensor corroboration information (e.g., information indicating that each of two or more cameras of the autonomous vehicle detected an object in the same video frame or at the same time).)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa such that the confidence is level if it was detected twice by two sensors with overlapping fields because if multiple sensors detected an obstacle, then it is more likely to be at where you see it than if only one sensor detected it. The redundancy reduces errors from false positives or malfunctioning sensors.
For Claim 9, Taylor teaches The safety system assembly according to claim 7
Taylor does not explicitly teach wherein the assessment device is configured to determine the confidence level for the object based on the quality of the sensor data and/or of the object information.
Moustafa, however, does teach wherein the assessment device is configured to determine the confidence level for the object based on the quality of the sensor data and/or of the object information. ([0297] FIG. 25 is a simplified diagram showing an example process of rating and validating crowdsourced autonomous vehicle sensor data in accordance with at least one embodiment. In the example shown, each autonomous vehicle 2502 collects data from one or more sensors coupled thereto (e.g., camera(s), LIDAR, radar, etc.). The autonomous vehicles 2502 may use the sensor data to control one or more aspects of the autonomous vehicle. As each autonomous vehicle collects data from its one or more sensors, the autonomous vehicle may determine an amount of confidence placed in datum collected. For example, the confidence score may be based on information related to the collection of the sensor data, such as, for example, weather data at the time of data collection (e.g., camera information on a sunny day may get a larger confidence score than cameras on a foggy day), sensor device configuration information (e.g., a bitrate or resolution of the camera stream), sensor device operation information (e.g., bit error rate for a camera stream), sensor device authentication status information (e.g., whether the sensor device has been previously authenticated by the autonomous vehicle, as described further below), or local sensor corroboration information (e.g., information indicating that each of two or more cameras of the autonomous vehicle detected an object in the same video frame or at the same time).)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa such that the confidence is based on the quality of sensor data or object information because if the sensor data is high quality, than the generated information may also be expected to be high quality. If the quality is expected to be low, then the precision or accuracy of the information may be suspect, and the confidence than an object is located at a particular location may be lower. This would allow the vehicle system to response to highly confident obstacles, and steer clear of vaguely defined obstacles.
For Claim 10, Taylor teaches The safety system assembly according to claim 9
Taylor does not explicitly teach wherein the quality of the:
a) sensor data depends on physical properties of the respective monitoring unit;
and/or
b) object information depends on the position, direction of movement and/or object class.
Moustafa, however, does teach wherein the quality of the:
a) sensor data depends on physical properties of the respective monitoring unit; ([0297] FIG. 25 is a simplified diagram showing an example process of rating and validating crowdsourced autonomous vehicle sensor data in accordance with at least one embodiment. In the example shown, each autonomous vehicle 2502 collects data from one or more sensors coupled thereto (e.g., camera(s), LIDAR, radar, etc.). The autonomous vehicles 2502 may use the sensor data to control one or more aspects of the autonomous vehicle. As each autonomous vehicle collects data from its one or more sensors, the autonomous vehicle may determine an amount of confidence placed in datum collected. For example, the confidence score may be based on information related to the collection of the sensor data, such as, for example, weather data at the time of data collection (e.g., camera information on a sunny day may get a larger confidence score than cameras on a foggy day), sensor device configuration information (e.g., a bitrate or resolution of the camera stream), sensor device operation information (e.g., bit error rate for a camera stream), sensor device authentication status information (e.g., whether the sensor device has been previously authenticated by the autonomous vehicle, as described further below), or local sensor corroboration information (e.g., information indicating that each of two or more cameras of the autonomous vehicle detected an object in the same video frame or at the same time).)
and/or
b) object information depends on the position, direction of movement and/or object class.
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa such that the confidence is based on the physical properties of the sensor because if the sensor data is high output in high quality, or it is expected to be high quality information, than the generated information may also be expected to be high quality. If the physical characteristics of the sensors are poor, the data produced is expected to be low, then the precision or accuracy of the information may be suspect, and the confidence than an object is located at a particular location may be lower. This would allow the vehicle system to response to highly confident obstacles, and steer clear of vaguely defined obstacles.
For Claim 11, Taylor teaches The safety system assembly according to claim 10,
Taylor does not explicitly teach wherein the physical properties comprise the age, the type, the error rate, the failure rate, the scatter rate, the measurement method, the installation location and/or confidence information of the respective monitoring unit.
Moustafa, however, does teach wherein the physical properties comprise the age, the type, the error rate, the failure rate, the scatter rate, the measurement method, the installation location and/or confidence information of the respective monitoring unit. ([0297] FIG. 25 is a simplified diagram showing an example process of rating and validating crowdsourced autonomous vehicle sensor data in accordance with at least one embodiment. In the example shown, each autonomous vehicle 2502 collects data from one or more sensors coupled thereto (e.g., camera(s), LIDAR, radar, etc.). The autonomous vehicles 2502 may use the sensor data to control one or more aspects of the autonomous vehicle. As each autonomous vehicle collects data from its one or more sensors, the autonomous vehicle may determine an amount of confidence placed in datum collected. For example, the confidence score may be based on information related to the collection of the sensor data, such as, for example, weather data at the time of data collection (e.g., camera information on a sunny day may get a larger confidence score than cameras on a foggy day), sensor device configuration information (e.g., a bitrate or resolution of the camera stream), sensor device operation information (e.g., bit error rate for a camera stream), sensor device authentication status information (e.g., whether the sensor device has been previously authenticated by the autonomous vehicle, as described further below), or local sensor corroboration information (e.g., information indicating that each of two or more cameras of the autonomous vehicle detected an object in the same video frame or at the same time).)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa such that the physical properties include failure rate because a sensor with a high failure rate is more likely to provide inaccurate data that the system may want to treat with less confidence. If the failure rate is very low, the system may choose to treat the sensor data with much more confidence. This would allow the system to make more informed decisions in determining which objects are well defined and precisely located against ones that are poorly defined and have an unknown precise location.
For Claim 12, Taylor teaches The safety system assembly according to claim 10, wherein the object information depends on whether it is a person or an autonomously driving vehicle. ([0145] Further, in some examples, the control system may be configured to classify detected objects into a plurality of categories and determine the first or second threshold distance for each object based on the category into which the object is classified. Classification may be performed using, for example, a machine learning algorithm (e.g., artificial neural networks applied to images from a ToF camera). For example, objects may be generally classified into fixed objects (e.g., walls, pallet racks) and non-fixed objects (e.g., boxes, pallets, vehicles), with fixed objects assigned smaller threshold distances than non-fixed objects. Object may also be classified into more specific categories, such as pallet jack, human, pallet, box, stationary robot, etc., depending on the contents of the environment in which the vehicle is operating.
[0039] A buffer region may be placed around each of the objects identified as an obstacle. The buffer region may operate to enforce a minimum distance (i.e., a first threshold distance) away from the obstacle at which the vehicle is to stop to avoid colliding with the obstacle. The minimum distance may be based on a size of the obstacle, a type or classification of the obstacle, a speed of the vehicle, a size of the vehicle, a load carried by the vehicle, and/or a task assigned to the vehicle, among other factors.
For Claim 13, Taylor teaches The safety system assembly according to claim 7
wherein the assessment device comprises an Al module and wherein the Al module is configured to determine the classification level for an object based on the sensor data and/or the object information. ([0145] Further, in some examples, the control system may be configured to classify detected objects into a plurality of categories and determine the first or second threshold distance for each object based on the category into which the object is classified. Classification may be performed using, for example, a machine learning algorithm (e.g., artificial neural networks applied to images from a ToF camera). For example, objects may be generally classified into fixed objects (e.g., walls, pallet racks) and non-fixed objects (e.g., boxes, pallets, vehicles), with fixed objects assigned smaller threshold distances than non-fixed objects. Object may also be classified into more specific categories, such as pallet jack, human, pallet, box, stationary robot, etc., depending on the contents of the environment in which the vehicle is operating.)
Taylor does not teach wherein the assessment device comprises an Al module and wherein the Al module is configured to determine the confidence level for an object based on the sensor data and/or the object information.
Moustafa, however, does teach wherein the assessment device comprises a system and wherein the system is configured to determine the confidence level for an object based on the sensor data and/or the object information.
([0297] FIG. 25 is a simplified diagram showing an example process of rating and validating crowdsourced autonomous vehicle sensor data in accordance with at least one embodiment. In the example shown, each autonomous vehicle 2502 collects data from one or more sensors coupled thereto (e.g., camera(s), LIDAR, radar, etc.). The autonomous vehicles 2502 may use the sensor data to control one or more aspects of the autonomous vehicle. As each autonomous vehicle collects data from its one or more sensors, the autonomous vehicle may determine an amount of confidence placed in datum collected. For example, the confidence score may be based on information related to the collection of the sensor data, such as, for example, weather data at the time of data collection (e.g., camera information on a sunny day may get a larger confidence score than cameras on a foggy day), sensor device configuration information (e.g., a bitrate or resolution of the camera stream), sensor device operation information (e.g., bit error rate for a camera stream), sensor device authentication status information (e.g., whether the sensor device has been previously authenticated by the autonomous vehicle, as described further below), or local sensor corroboration information (e.g., information indicating that each of two or more cameras of the autonomous vehicle detected an object in the same video frame or at the same time).)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa such that an AI module determines a confidence level for the sensor data for objects because it is a task that an AI module would be expected to be successful at, as examples could be provided to a machine learning system. It would then be capable of effectively determining the confidence which could be provided to the autonomous system so that it can respond appropriately to indications that there is a high confidence of an object or obstacle at a location, or a low confidence of this.
For Claim 14, Taylor teaches The safety system assembly according to claim 7
Taylor does not teach wherein the safety system assembly comprises at least one autonomously driving vehicle, wherein the at least one autonomously driving vehicle is configured to receive the object list from the central processing device.
Moustafa, however, does teach wherein the safety system assembly comprises at least one autonomously driving vehicle, wherein the at least one autonomously driving vehicle is configured to receive the object list from the central processing device.
[0226] Turning to FIG. 10, a simplified block diagram 1000 is shown illustrating an enhanced autonomous driving system including blocks (e.g., 1005, 1035, 1040, 244, etc.) providing functionality to intelligently manage the creation, storage, and offloading of sensor data generated by a sensor array 1005 on a corresponding autonomous vehicle. For instance, the sensor array 1005 may be composed of multiple different types of sensors (such as described herein) and may be further provided with pre-processing software and/or hardware to perform some object recognition and provide object list results as well as raw data. In some implementations, the pre-processing logic may also assist in optimizing data delivery and production. Data from the sensor array 1005 may be provided to an in-vehicle data reservoir 1010 (or memory), which may be accessed and used by other functional blocks of the autonomous driving system. For instance, an autonomous driving stack 1015 using various artificial intelligence logic and machine learning models may receive or retrieve the sensor data to generate outputs to the actuation and control block 1020 to autonomously steer, accelerate, and brake the vehicle 105. In some cases, results generated by the autonomous driving stack 1015 may be shared with other devices (e.g., 1025) extraneous to the vehicle 105.
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa such that there is an autonomous vehicle that receives the object list because if the control decisions are made on the autonomous vehicle, then the object list and the corresponding information could be useful for making control decisions. These decisions could include which objects to avoid to prevent collisions, which would be useful at preventing harm and damage.
For Claim 18, Taylor teaches The safety system assembly according to claim 14
Taylor does not teach wherein the at least one autonomously driving vehicle is configured, even in the event of a malfunction of at least one safety system that serves to monitor the environment, to continue travelling if the objects on the object list do not lead to a collision and the confidence level of these objects is greater than a threshold value. ([0222] In some instances, it may take only moments from the status of one or more sensors to degrade from operable to compromised. For instance, it can take but a moment for a mud splash, an unexpected malfunction, poorly angled headlight or sunlight, etc. to compromise the integrity of a sensor. Transitioning to a recommender-recognized, lesser driving autonomy mode in that moment may be unduly demanding from an operational and practical perspective. Accordingly, in some implementations, a recommender system of a vehicle (e.g., 105) may possess predictive analytics and machine learning logic to predict instances where the use of the recommender system (and supplemental sensor and object recognition data) or change in autonomy level is more likely. For instance, one or more machine learning models may be provided to accept inputs from systems of the vehicle (e.g., 105) to predict when a trip is more likely to rely on such changes to the operation of the vehicle's higher level autonomous driving functionality. For instance, inputs may include sensor status information, system diagnostic information, sensor age or use statistics, weather information (e.g., which may result in sloppy roads, salt-treated roads, reduced visibility, etc.), road condition information (e.g., corresponding to roads that are more likely to hamper functioning of the sensors (e.g., dirt roads, roads with standing water, etc.), among other inputs. If such a likelihood is predicted, the vehicle may prepare systems (e.g., preemptively begin accepting data from other devices describing the current or upcoming stretches of road along a planned route) in case an issue arises with one or more of the sensors, allowing the vehicle to react promptly to a sensor issue and adjust operation of its autonomous driving logic, among other example implementations.
[0298] The autonomous vehicle may calculate a confidence score, which may be maintained in metadata associated with the data. The confidence score may be a continuous scale between zero and one in some implementations (rather than a binary decision of trusting everything or trusting nothing), or between zero and another number (e.g., 10). Additionally, in cases where the collection device is capable of authentication or attestation (e.g., where the device is authenticated by the autonomous vehicle before the autonomous vehicle accepts the data from the device), the device's authentication/attestation status may be indicated in the metadata of the data collected by the sensor device (e.g., as a flag, a digital signature, or other type of information indicating the authentication status of the sensor device), allowing the server 2504 or other autonomous vehicle to more fully verify/validate/trust the data before using the data to update the HD map. In some cases, the autonomous vehicle itself may be authenticated (e.g., using digital signature techniques) by the server. In such cases, the data collected from different sensors of the autonomous vehicle may be aggregated, and in some cases authenticated, by the main processor or processing unit within the autonomous vehicle before being transferred or otherwise communicated to the server or to nearby autonomous vehicles.)
Therefore, it would be obvious to one of ordinary skill in the art prior to effective filing date to modify Taylor in light of Moustafa such that if a safety system is broken the system still continues if confidence is high because not all sensors may be necessary for the vehicle to function if the other sensors are able to detect obstacles around the object. Rather than halt operations when a redundant sensor is broken, the sensor can be repaired later when operations have ceased.
For Claim 19, Taylor teaches A method of monitoring a zone, in which objects move together, comprising the following method steps:
- receiving sensor data from a plurality of monitoring units, in which sensor data the objects detected in the monitored zone by the monitoring units are included; (([0112] In some embodiments, the map may be determined based on data from sensor 806, other sensors on other vehicles within the environment, or other sensors positioned at fixed locations within the environment. The map may represent a fusion of information from multiple different sensors, or from multiple different types of sensors. In some examples, the map may be a two-dimensional representation of the environment, and may be determined or updated by projecting representations of physical features within the environment detected by the sensors onto a plane defined by the two-dimensional representation of the environment. The map may represent a threshold extent of the environment around the vehicle (e.g., a 10 meter by 10 meter square centered around the vehicle). Alternatively, the map may represent the environment in three dimensions, and may therefore represent a volume of space centered around the vehicle.))
- consolidating the received sensor data; (([0112] In some embodiments, the map may be determined based on data from sensor 806, other sensors on other vehicles within the environment, or other sensors positioned at fixed locations within the environment. The map may represent a fusion of information from multiple different sensors, or from multiple different types of sensors. In some examples, the map may be a two-dimensional representation of the environment, and may be determined or updated by projecting representations of physical features within the environment detected by the sensors onto a plane defined by the two-dimensional representation of the environment. The map may represent a threshold extent of the environment around the vehicle (e.g., a 10 meter by 10 meter square centered around the vehicle). Alternatively, the map may represent the environment in three dimensions, and may therefore represent a volume of space centered around the vehicle.))
Taylor does not explicitly teach - creating object lists from the consolidated sensor data, wherein the object lists include the detected objects together with the respective object information;
- transmitting these object lists to the autonomously driving vehicles.
Moustafa, however, does teach- creating object lists from the consolidated sensor data, wherein the object lists include the detected objects together with the respective object information;
- transmitting these object lists to the autonomously driving vehicles.
([0226] Turning to FIG. 10, a simplified block diagram 1000 is shown illustrating an enhanced autonomous driving system including blocks (e.g., 1005, 1035, 1040, 244, etc.) providing functionality to intelligently manage the creation, storage, and offloading of sensor data generated by a sensor array 1005 on a corresponding autonomous vehicle. For instance, the sensor array 1005 may be composed of multiple different types of sensors (such as described herein) and may be further provided with pre-processing software and/or hardware to perform some object recognition and provide object list results as well as raw data. In some implementations, the pre-processing logic may also assist in optimizing data delivery and production. Data from the sensor array 1005 may be provided to an in-vehicle data reservoir 1010 (or memory), which may be accessed and used by other functional blocks of the autonomous driving system. For instance, an autonomous driving stack 1015 using various artificial intelligence logic and machine learning models may receive or retrieve the sensor data to generate outputs to the actuation and control block 1020 to autonomously steer, accelerate, and brake the vehicle 105. In some cases, results generated by the autonomous driving stack 1015 may be shared with other devices (e.g., 1025) extraneous to the vehicle 105.)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Moustafa so that object lists with information are created and sent to the autonomous vehicle, because it would allow the autonomous vehicle to take corrective action to avoid colliding with the obstacles, especially if there are control elements and controllers making decisions on the autonomous vehicle itself.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Taylor in light of Moustafa in light of Chu et al (US Pub 2018/0348771 A1) hereafter referred to as Chu.
For Claim 15, Taylor teaches The safety system assembly according to claim 14
to drive into the intersection zone at a reduced speed or to stop if objects on the object list whose distance from the intersection zone is smaller than a distance threshold value ([0120] FIG. 8C illustrates object 808c, with which vehicle 800 is predicted to collide, surrounded by a buffer 826 (i.e., an obstacle buffer). The buffer may be a circle having a radius equal to the threshold distance (i.e., a first threshold distance or a collision threshold distance). Alternatively, in some embodiments, the buffer may have a shape similar to the shape of the first object and may be offset from a perimeter thereof by the threshold distance. Thus, the buffer around object 808c, for example, might be a square circumscribed around the circle making up buffer 826. The buffer may be applied to any obstacles with which at least one footprint is determined to intersect.
[0121] In some embodiments, the first threshold distance may be equal to or greater than a maximum distance needed for vehicle 800 to come to a stop, and may thus be based on physical and kinematic properties of vehicle 800 (e.g., speed, mass, load carried, etc.). Buffer 826 may thus allow vehicle 800 to be stopped short of obstacle 808c.)
Taylor does not teach wherein the at least one autonomously driving vehicle is configured to:
a) drive into an intersection zone without braking if objects on the object list whose distance from the intersection zone is smaller than a distance threshold value have a confidence level that is greater than the first threshold value; and
b) to drive into the intersection zone at a reduced speed or to stop if objects on the object list whose distance from the intersection zone is smaller than a distance threshold value have a confidence level that is smaller than a second threshold value.
Chu, however, does teach wherein the at least one autonomously driving vehicle is configured to:
a) drive into an intersection zone without braking if objects on the object list have a confidence level that is greater than the first threshold value; and
b) to drive into the intersection zone at a reduced speed or to stop if objects on the object list have a confidence level that is smaller than a second threshold value.
([0078] Task 634 determines whether the confidence in the last known good component is above a confidence threshold. In the example provided, the confidence threshold is an amount of time since the last known good component was calculated. In the example provided, the plan validity is based on the uncertainty of the prediction horizon. For example, the confidence decreases faster in time when the object is moving fast and/or when the object was only partially tracked. For example, stop contingency system 100 may determine that the last known good component is below the predetermined confidence threshold when the last known good component was calculated more than five seconds before executing task 634. When the confidence in the last known component is above the confidence threshold, method 600 proceeds to task 628. When the confidence in the last known component is below the confidence threshold, method 600 proceeds to task 636.
[0079] Task 636 commands the vehicle to execute a hard stop in response to determining that a component confidence in one of the previous lateral component and the previous longitudinal component is below a predetermined confidence threshold. For example, plan implementation module 420 may instruct the vehicle 10 to apply a maximum braking force available in task 636.
[0058] In various embodiments, the motion planning module 410 receives sensor data 412 from various sensors 40a-40n of the vehicle 10 (e.g., lidar sensors, radar sensors, cameras, and so on). The motion planning module 410 gathers the sensor data 412 in order to obtain information pertaining to one or more potential obstacles in proximity to the vehicle 10, to an environment surrounding the vehicle 10, and to the availability and health of various vehicle systems. In various embodiments, the sensor data 412 is obtained via the sensors 40a-40n of FIG. 1. In various embodiments, the sensor data 412 may include, among other data, a type of potential obstacle (e.g., another vehicle, a pedestrian, an animal), information as to whether the potential obstacle is moving, usage of the brakes and signals (e.g., blinkers) when the potential obstacle is a vehicle, a lane position of the potential obstacle, and presence of a traffic intersection proximate the potential obstacle, among other possible information. In some embodiments, the motion planning module 410 similarly obtains other data as part of the sensor data 412, such as passenger inputs (e.g., as to a desired destination) and/or remote data from sources outside the vehicle 10 (e.g., from GPS systems, traffic providers, and so on). In various embodiments, the motion planning module 410 gathers this information and generates driving plan data 415 as outputs for the motion planning module 410, which are provided to the plan implementation module 420 described below.)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Chu such wherein the at least one autonomously driving vehicle is configured to:
a) drive into an intersection zone without braking if objects on the object list whose distance from the intersection zone is smaller than a distance threshold value have a confidence level that is greater than the first threshold value; and
b) to drive into the intersection zone at a reduced speed or to stop if objects on the object list whose distance from the intersection zone is smaller than a distance threshold value have a confidence level that is smaller than a second threshold value.
It would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Chu this way because if the system does not know the exact location of a potential obstacle, then it does not know if it may collide with that obstacle if it continues towards it. By slowing down or braking, it is ensuring that it lessens the damage of a collision, or reduces the chance of one occurring. If the system knows exactly where an obstacle is, it can plan accordingly, but if it does not then moving towards an intersection that the obstacle is associated with could result in a collision, which could cause harm and damage.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Taylor in light of Moustafa in light of Chu in light of Vrba et al (US Pub 2024/0101174 A1) hereafter known as Vrba.
For Claim 16, Taylor teaches The safety system assembly according to claim 15
Taylor does not teach wherein the at least one autonomously driving vehicle is configured to bypass or deactivate at least one or all of the safety systems of the autonomously driving vehicle in the event that a driving into the intersection zone without braking takes place.
Vrba, however, does teach wherein the at least one autonomously driving vehicle is configured to bypass or deactivate at least one or all of the safety systems of the autonomously driving vehicle in the event that the environment is determined to have a sufficiently low density of traffic and obstacles with high confidence. ([0030] The type of cargo being carried by the vehicle can indicate whether the vehicle is carrying hazardous cargo (explosive, corrosive, poisonous, or radioactive materials) that poses a health risk to people, livestock, or vegetation, is carrying people, is not carrying any of the aforementioned types of cargo (non-person and non-hazardous cargo), or is not carrying any cargo. The operational mode may be selected as the restricted mode while the vehicle is carrying hazardous cargo or people. This can restrict the vehicle to travel slower and/or be able to stop faster in areas where there is increased risk of damage to persons, livestock, or vegetation if an accident were to occur and the cargo dumped off the vehicle. The operational mode may be selected as the unrestricted mode while the vehicle is not carrying hazardous cargo, is not carrying people, and/or is not carrying any cargo.
[0031] The population density may be the density of persons living or currently within a geographic area in which the vehicle is located or expected/scheduled to travel. The operational mode may be selected as the restricted mode responsive to the vehicle being in or expected to travel through an area with a population density that exceeds a population density threshold. This can restrict the vehicle to travel slower and/or be able to stop faster in areas where there is increased risk of collisions with people. The operational mode may be selected as the unrestricted mode responsive to the vehicle not being located in and/or not expected to travel through an area with a population density that exceeds the population density threshold.
[0032] The traffic density may be the density of other vehicles within a geographic area in which the vehicle is located or expected/scheduled to travel. The operational mode may be selected as the restricted mode responsive to the vehicle being in or expected to travel through an area with a traffic density that exceeds a traffic density threshold. This can restrict the vehicle to travel slower and/or be able to stop faster in areas where there is increased risk of collisions with people. The operational mode may be selected as the unrestricted mode responsive to the vehicle not being located in and/or not expected to travel through an area with a traffic density that exceeds the traffic density threshold.)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Vrba so that wherein the at least one autonomously driving vehicle is configured to bypass or deactivate at least one or all of the safety systems of the autonomously driving vehicle in the event that a driving into the intersection zone without braking takes place.
It would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of Vrba in this way because if the system is certain that there are no obstacles or pedestrians that may be harmed in the general vicinity of the work area, then safety features that automatically stop the vehicle when objects are detected may interfere with trajectories through tight areas in which every object is known and static. This would allow the system to navigate through tight areas without proximity sensors stopping the vehicle, since the system is certain no unexpected obstacles will be present.
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Taylor in light of Moustafa in light of Chu in light of Vrba in light of Wicks et al (US Pub 2015/0352721 A1) hereafter known as Wicks in light of Ross et al (US Pub 2023/0042650 A1), hereafter known as Ross..
For Claim 17, Taylor teaches The safety system assembly according to claim 14
wherein the at least one autonomously driving vehicle is configured to drive into a warehouse in order to unload or load goods, ([0001] A warehouse may be used for storage of goods by a variety of different types of commercial entities, including manufacturers, wholesalers, and transport businesses. Example stored goods may include raw materials, parts or components, packing materials, and finished products. In some cases, the warehouse may be equipped with loading docks to allow goods to be loaded onto and unloaded from delivery trucks or other types of vehicles. The warehouse may also use rows of pallet racks to allow for storage of pallets, which are flat transport structures that contain stacks of boxes or other objects thereon. Additionally, the warehouse may use machines or vehicles for lifting and moving goods or pallets of goods, such as cranes, forklifts, and pallet jacks. Human operators may be employed to operate machines, vehicles, and other equipment. In some cases, one or more of the machines or vehicles may be robotic devices guided by computer control systems.)
Taylor does not teach wherein the at least one autonomously driving vehicle is configured to drive into a truck and/or a railroad car in order to unload or load goods, wherein the autonomously driving vehicle only drives into the truck and/or the railroad car if, in a distance range around the truck and/or the railroad car that is smaller than a distance threshold value, the objects on the object list:
a) are not persons;
b) the corresponding object information of the objects has a confidence level that is greater than a threshold value;
wherein the autonomously driving vehicle is configured to bypass or switch off one or all of the safety systems when driving into the truck and/or the railroad car.
Wicks, however, does teach wherein the at least one autonomously driving vehicle is configured to drive into a truck and/or a railroad car in order to unload or load goods, ([0040] FIG. 1 illustrates an embodiment communication system 100 that includes an exemplary robotic carton unloader 101 configured with a plurality of sensor devices 102, 104, 106. The robotic carton unloader 101 may be a mobile vehicle, such as a platform (or frame) with wheels capable of moving through an unloading area 120 (or cargo area). For example, the robotic carton unloader 101 may be designed for use within the trailer of a cargo truck 121. In some embodiments, distance sensors (not shown) may be utilized by the robotic carton unloader 101 to guide the robotic carton unloader 101 into the unloading area 120 (e.g., a semi-trailer). For example, the robotic carton unloader 101 may utilize “curb feeler” distance sensors that use contact to measure distance from the walls of the unloading area 120. Alternately, such distance sensors may use light, sound, or other methods to sense distance.)
Ross, however, does teach stopping the robot if a human is a threshold distance near the robot. [0091] According to at least one non-limiting exemplary embodiment, robot 102 may halt navigation along route 302 upon detecting the human 314 and disable or modify the emission pattern of the UV light. The robot 102 may continue navigation upon the human 314 moving a threshold distance from the robot 102 or moving beyond the emission pattern 306 such that the robot 102 disinfects the entire surface of objects 304.
Chu, however, does teach
wherein the autonomously driving vehicle only drives if
b) the corresponding object information of the objects has a confidence level that is greater than a threshold value;
([0078] Task 634 determines whether the confidence in the last known good component is above a confidence threshold. In the example provided, the confidence threshold is an amount of time since the last known good component was calculated. In the example provided, the plan validity is based on the uncertainty of the prediction horizon. For example, the confidence decreases faster in time when the object is moving fast and/or when the object was only partially tracked. For example, stop contingency system 100 may determine that the last known good component is below the predetermined confidence threshold when the last known good component was calculated more than five seconds before executing task 634. When the confidence in the last known component is above the confidence threshold, method 600 proceeds to task 628. When the confidence in the last known component is below the confidence threshold, method 600 proceeds to task 636.
[0079] Task 636 commands the vehicle to execute a hard stop in response to determining that a component confidence in one of the previous lateral component and the previous longitudinal component is below a predetermined confidence threshold. For example, plan implementation module 420 may instruct the vehicle 10 to apply a maximum braking force available in task 636.
[0058] In various embodiments, the motion planning module 410 receives sensor data 412 from various sensors 40a-40n of the vehicle 10 (e.g., lidar sensors, radar sensors, cameras, and so on). The motion planning module 410 gathers the sensor data 412 in order to obtain information pertaining to one or more potential obstacles in proximity to the vehicle 10, to an environment surrounding the vehicle 10, and to the availability and health of various vehicle systems. In various embodiments, the sensor data 412 is obtained via the sensors 40a-40n of FIG. 1. In various embodiments, the sensor data 412 may include, among other data, a type of potential obstacle (e.g., another vehicle, a pedestrian, an animal), information as to whether the potential obstacle is moving, usage of the brakes and signals (e.g., blinkers) when the potential obstacle is a vehicle, a lane position of the potential obstacle, and presence of a traffic intersection proximate the potential obstacle, among other possible information. In some embodiments, the motion planning module 410 similarly obtains other data as part of the sensor data 412, such as passenger inputs (e.g., as to a desired destination) and/or remote data from sources outside the vehicle 10 (e.g., from GPS systems, traffic providers, and so on). In various embodiments, the motion planning module 410 gathers this information and generates driving plan data 415 as outputs for the motion planning module 410, which are provided to the plan implementation module 420 described below.)
Vrba, however, does teach wherein the autonomously driving vehicle is configured to bypass or switch off one or all of the safety systems when driving. ([0030] The type of cargo being carried by the vehicle can indicate whether the vehicle is carrying hazardous cargo (explosive, corrosive, poisonous, or radioactive materials) that poses a health risk to people, livestock, or vegetation, is carrying people, is not carrying any of the aforementioned types of cargo (non-person and non-hazardous cargo), or is not carrying any cargo. The operational mode may be selected as the restricted mode while the vehicle is carrying hazardous cargo or people. This can restrict the vehicle to travel slower and/or be able to stop faster in areas where there is increased risk of damage to persons, livestock, or vegetation if an accident were to occur and the cargo dumped off the vehicle. The operational mode may be selected as the unrestricted mode while the vehicle is not carrying hazardous cargo, is not carrying people, and/or is not carrying any cargo.
[0031] The population density may be the density of persons living or currently within a geographic area in which the vehicle is located or expected/scheduled to travel. The operational mode may be selected as the restricted mode responsive to the vehicle being in or expected to travel through an area with a population density that exceeds a population density threshold. This can restrict the vehicle to travel slower and/or be able to stop faster in areas where there is increased risk of collisions with people. The operational mode may be selected as the unrestricted mode responsive to the vehicle not being located in and/or not expected to travel through an area with a population density that exceeds the population density threshold.
[0032] The traffic density may be the density of other vehicles within a geographic area in which the vehicle is located or expected/scheduled to travel. The operational mode may be selected as the restricted mode responsive to the vehicle being in or expected to travel through an area with a traffic density that exceeds a traffic density threshold. This can restrict the vehicle to travel slower and/or be able to stop faster in areas where there is increased risk of collisions with people. The operational mode may be selected as the unrestricted mode responsive to the vehicle not being located in and/or not expected to travel through an area with a traffic density that exceeds the traffic density threshold.)
Therefore, it would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor such that wherein the at least one autonomously driving vehicle is configured to drive into a truck and/or a railroad car in order to unload or load goods, wherein the autonomously driving vehicle only drives into the truck and/or the railroad car if, in a distance range around the truck and/or the railroad car that is smaller than a distance threshold value, the objects on the object list:
a) are not persons;
b) the corresponding object information of the objects has a confidence level that is greater than a threshold value;
wherein the autonomously driving vehicle is configured to bypass or switch off one or all of the safety systems when driving into the truck and/or the railroad car.
It would be obvious to one of ordinary skill in the art prior to the effective filing date to modify Taylor in light of the cited art to do this because goods are frequently brought to factories and warehouses on trucks and railcars, and being able to drive inside the truck or railcar would allow the vehicle to remove it without needing a human to climb up and remove the goods themselves. Additionally, by ensuring that there are no people within the area, and only objects with high confidence of their location, the autonomous vehicle can move into a tight, restrained area without worrying about movement, unexpected obstacles, or injuring humans. By disabling some or all of the safety features, the system can navigate in tight area without having to worry about automatically stopping when it gets too close to a wall or boundary.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Batts et al (US Pub 2020/0339151 A1) relates to vehicle systems that continue despite sensor failure.
Delhaye et al (US Pub 2022/0227360 A1) relates to braking vehicles intersections considering confidence data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to TRISTAN J GREINER whose telephone number is (571)272-1382. The examiner can normally be reached Mon - Fri 7:30-4:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tran Khoi can be reached at Monday-Thursday. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/T.J.G./Examiner, Art Unit 3656
/KHOI H TRAN/Supervisory Patent Examiner, Art Unit 3656
Prosecution Insights