DETAILED ACTION
In view of the appeal brief filed on 1/16/2026, PROSECUTION IS HEREBY REOPENED. A new ground of rejection is set forth below.
To avoid abandonment of the application, appellant must exercise one of the following two options:
(1) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply under 37 CFR 1.113 (if this Office action is final); or,
(2) initiate a new appeal by filing a notice of appeal under 37 CFR 41.31 followed by an appeal brief under 37 CFR 41.37. The previously paid notice of appeal fee and appeal brief fee can be applied to the new appeal. If, however, the appeal fees set forth in 37 CFR 41.20 have been increased since they were previously paid, then appellant must pay the difference between the increased fees and the amount previously paid.
A Supervisory Patent Examiner (SPE) has approved of reopening prosecution by signing below:
Claims 1-26 are pending in the application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Response to Arguments
Regarding the Applicants’ Argument A that “Appellant respectfully submits that the combination of references does not teach or suggest, at least,
"based at least on the analyzing the first content, determining one or more values that identify one or more instances of the one or more time intervals, the one or more values indicating portions of the searchable data that fail to satisfy the initial query," and "converting the initial query into an updated query comprising a logical expression that defines a second search space and the scenario using the one or more values and the one or more relationships to prune, from the first search space, the portions of the searchable data," as recited in amended claim 1”, please see the new combination of references cited below.
Specification, para. 5 states the current application relates “to a query engine for designing, analyzing, optimizing, verifying, and/or validating simulations and simulated data in autonomous machine applications”, that “allow for searching of test data - including real-world data, simulation data, system under test (SUT) data, and/or map data”.
Cahoon teaches at col. 4:3-38: using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated. Coordinate systems can be utilized to describe positions and/or speeds (or velocities) of entities in a scenario; col. 8:2-30: examples of variables can include a position in a lane on a road, a time, surface conditions, etc.; col. 9:1-23: consider the scenario with respect to figs. 2 and 3. In such a scenario, the "wait" condition is associated with a value of one second. However, to achieve the combinatorial combinations described above, the time associated with the "wait" condition can be varied. For instance, the time can be a value of one second or five seconds. A second variable in the scenario can correspond to the speed (or velocity) of the observer car. For instance, the speed (or velocity) of the observer car can be varied. The observer car can be travelling at 10 miles-per-hour (mph) or 30 mph).
Thus, as map data is updated responding to the vehicle is moving into a new location/operational space, the past location/searchable data is no longer satisfied the initial search/query. The map or current operational space of the autonomous vehicle, e.g., location, road, objects, lines etc. are changed as the vehicle moves to different location, thus, searches are automatically updated with new values in order to control the operations of the vehicle.
In addition, Cahoon teaches at col. 4:3-54: the SDL enables running an "exhaustive enumerative" search over conditions, as well as enabling a user to run tests often as they develop a system. Coordinate systems can be utilized to describe positions and/or speeds (or velocities) of entities in a scenario. Coordinate systems can include inertial coordinate systems, track based coordinate systems, map-based coordinate systems. Conditions can also include Boolean operators, such as, but not limited to, and, or not, xor, nor, nand, etc, such that multiple conditions as primitives can be combined;
col. 10:43-47: the autonomous controller corresponds to software that is also run on autonomous vehicles, as described above. Accordingly, a response of the autonomous controller can represent how an autonomous vehicle is likely to respond in a real environment.
Thus, as map data is updated responding to the vehicle moves to a new location, operational space and time, the prior values in relating to the vehicle/searchable data in past time period are no longer satisfied or inherently different from the vehicle’s new location and surrounding objects. Therefore, the system continuously updates the search on searchable data/map data, test data etc. to the current operational space of the autonomous vehicle location, e.g., road, objects, lines etc. or values to continuously control the vehicle. Therefore, the vehicle past location, operational search space, values, relationships etc. are no longer used/pruned to use the appropriate/current searchable data for continuously control the vehicle in the new location and relating environment.
Please also see the newly cited reference: Testke belo
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claim 5 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. The limitation “same solution space” is not in the specification.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-11, 13-14, 16-20, and 22-25 are rejected under 35 U.S.C. 103 as being unpatentable over Cahoon et al. (US 10489529) in view of Zhou (US 20190130188) and further in view of Teske (US 20190383637).
Specification, para. 5 states the current application relates “to a query engine for designing, analyzing, optimizing, verifying, and/or validating simulations and simulated data in autonomous machine applications”, that “allow for searching of test data - including real-world data, simulation data, system under test (SUT) data, and/or map data”.
As per claims 1, 11, 18, Cahoon et al. teaches
a system comprising: one or more processing units to execute operations comprising: receiving data representative of an initial query, the initial query defining a first search space and a scenario including one or more relationships in time or space between one or more objects and an ego-machine over one or more time intervals (col. 2:13-67: simulations can be used to validate software (i.e., an autonomous controller) being run on autonomous vehicles to ensure that the software is able to safely control such autonomous vehicles; col. 4:8-65: the SDL/Scenario Description Language enables running an "exhaustive enumerative" search (equivalent to initial query searches from initial set of data: test data, map data etc.) over conditions, as well as enabling a user to run tests often as they develop a system; col. 5, line 47-col. 6, line 30; col. 8:3-9: a sequence instructs the simulator to first wait for the test car to approach the intersection, wait 1 second and then have the test car follow the observer car. Thus, the initial query is to search over conditions, e.g., location “approach the intersection”, a time interval “1 second” etc.);
analyzing, using the initial query, first content captured stored in searchable data to determine whether the first content satisfies the one or more relationships over the one or more time intervals (col. 4:7-65: using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated. The SDL enables running an "exhaustive enumerative" search over conditions (equivalent to search/query the map data/searchable data, test data, simulation data or real world data for conditions that match certain scenarios), as well as enabling a user to run tests often as they develop a system…a condition primitive can cause the performance of some action or evaluation of some other condition to be delayed until a condition associated with the condition primitive is satisfied. The SDL allows for easily defining scenarios at a high level to be constructed based on a set of primitives. Primitives can be instantiated within an environment, such as within a map. Conditions can also include Boolean operators, such as, but not limited to, and, or not, xor, nor, nand, etc, such that multiple conditions as primitives can be combined. A "wait" condition can be satisfied upon lapse of the period of time; col. 8:29-30: examples of variables can include a position in a lane on a road, a time, surface conditions, etc.; col. 9:24-col.10:47: consider an outer product where a user specifies a range of starting positions and a range of a number of other primitives (e.g., entities with associated dimensions, velocities, and categories) to be instantiated (equivalent to searchable data and relationship(s) between primitives over one or more time periods) limiting which scenarios are created for validation and testing can reduce a number of required computations);
based at least on the analyzing the first content, determining one or more values that identify one or more instances of the one or more time intervals, the one or more values indicating portions of the searchable data that fail to satisfy the initial query (col. 4:3-38: using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated. Coordinate systems can be utilized to describe positions and/or speeds (or velocities) of entities in a scenario; col. 8:2-30: examples of variables can include a position in a lane on a road, a time, surface conditions, etc.; col. 9:1-23: consider the scenario with respect to figs. 2 and 3. In such a scenario, the "wait" condition is associated with a value of one second. However, to achieve the combinatorial combinations described above, the time associated with the "wait" condition can be varied. For instance, the time can be a value of one second or five seconds. A second variable in the scenario can correspond to the speed (or velocity) of the observer car. For instance, the speed (or velocity) of the observer car can be varied. The observer car can be travelling at 10 miles-per-hour (mph) or 30 mph). Thus, as map data is updated responding to the tracked vehicle is moving into a new location/operational space, the past location/searchable data is no longer satisfied the initial search/query. The map or current operational space of the autonomous vehicle, e.g., location, road, objects, lines etc. are changed as the vehicle moves to different location, thus, searches are automatically updated with new values in order to control the operations of the vehicle.
converting the initial query into an updated query comprising a logical expression that defines a second search space and the scenario using the one or more values and the one or more relationships to prune, from the first search space, the portions of the searchable data (col. 4:3-54: using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated. As such, the SDL enables running an "exhaustive enumerative" search over conditions, as well as enabling a user to run tests often as they develop a system. Coordinate systems can be utilized to describe positions and/or speeds (or velocities) of entities in a scenario. Coordinate systems can include inertial coordinate systems, track based coordinate systems, map-based coordinate systems. Conditions can also include Boolean operators, such as, but not limited to, and, or not, xor, nor, nand, etc, such that multiple conditions as primitives can be combined; col. 10:43-47: the autonomous controller corresponds to software that is also run on autonomous vehicles, as described above. Accordingly, a response of the autonomous controller can represent how an autonomous vehicle is likely to respond in a real environment. Thus, as map data is updated responding to the vehicle moves to a new location, operational space and time, the prior values in relating to the vehicle/searchable data in past time period are no longer satisfied or inherently different from the current vehicle’s surrounding objects. Therefore, the system updates the search to the current operational space of the autonomous vehicle location, e.g., road, objects, lines etc. or values to continuously control the vehicle. The vehicle past location, operational search space, values, relationships etc. are no longer used/pruned to continuously control the vehicle in the new location and relating environment.)
Cahoon does not explicitly teach content captured in one or more frames of sequences of frames; analyzing, using the logical expression of the updated query, second content captured in the sequences of frames stored in the searchable data to determine at least one sub-sequence of frames of the sequences that satisfies the updated query; based at least on the analyzing the second content, generating query results to the initial query, the query results identifying that the at least one sub-sequence of frames captures the scenario.
Zhou et al. teaches
first content captured in one or more frames of sequences of frames stored in searchable data (para. 64: a video analytics system can obtain a sequence of video frames from a video source and can process the video sequence to perform a variety of tasks.; para. 67-69: video analytics can be trained to recognize certain objects. Another function that can be performed by video analytics includes providing demographics for customer metrics (e.g., customer counts, gender, age, amount of time spent, and other suitable metrics). Video analytics can also perform video search (e.g., extracting basic activity for a given region) and video summary (e.g., extraction of the key movements). The video analytics system receives video frames from a video source. The video frames can also be referred to herein as a video picture or a picture. The video frames can be part of one or more video sequences. The video source can include a video capture device (e.g., a video camera, a camera phone, a video phone, or other suitable capture device), a video storage device);
analyzing, using the logical expression of the updated query, second content captured in the sequences of frames stored in the searchable data to determine at least one sub-sequence of frames of the sequences that satisfies the updated query (para. 67: video analytics can operate as an Intelligent Video Motion Detector by detecting moving objects and by tracking moving objects; fig. 7: video analytics system analyzing received video frames. The object tracking system performs Blob tracking and updating system; para. 72-75: when the blob tracker is updated in the previous frame (after being associated with the previous blob in the previous frame), updated information for the blob tracker can include the tracking information for the previous frame and also prediction of a location of the blob tracker in the next frame (which is the current frame in this example). The prediction of the location of the blob tracker in the current frame can be based on the location of the blob in the previous frame. A history or motion model can be maintained for a blob tracker, including a history of various states, a history of the velocity, and a history of location, of continuous frames; para. 131);
based at least on the analyzing the second content, generating query results to the initial query, the query results identifying that the at least one sub-sequence of frames captures the scenario (para. 67: video analytics can also perform video search (e.g., extracting basic activity for a given region) and video summary (e.g., extraction of the key movements); para. 77: when a next video frame N 202N is received, the blob detection system 204N generates foreground blobs 208N for the frame N 202N. The object tracking system 206N can then perform temporal tracking of the blobs 208N. For example, the object tracking system 206N obtains the blob trackers 310A that were updated based on the prior video frame A; para. 98-100: once the blobs are detected and processed, object tracking (also referred to as blob tracking) can be performed to track the detected blobs. When blobs are detected from an input video frame, blob trackers from the previous video frame need to be associated to the blobs in the input video frame. The blob trackers can be updated based on the associated foreground blobs; para. 131-133: tracking a tracked object/blob when certain conditions are met (e.g., the blob has been tracked for a certain number of frames, a certain period of time, and/or other suitable conditions). The blob tracking and updating system 714 can also include a video analytics manager that can record object detection and tracking events; para. 217: fig. 22 - fig. 29 are video frames illustrating several subjective examples showing results of the video analytics with classification techniques).
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon and object/blob tracking to sequences of received data frames of Zhou in order to effectively analyze and identify scenarios in the captured video(s) to alert users of certain events or control objects based on scenarios as taught by Cahoon – See Cahoon, col. 2:13-67 and Zhou, para. 100: the bounding box of the blob tracker can be the bounding box of a blob associated with the blob tracker in a previous frame. Such techniques can be good for certain controlled scenarios, such as well-aligned lane conveying.
Even if Cahoon and Zhou do not explicitly teach the one or more values indicating portions of the searchable data that fail to satisfy the initial query; converting the initial query into an updated query comprising a logical expression that defines a second search space and the scenario using the one or more values and the one or more relationships to prune, from the first search space, the portions of the searchable data,
Teske teaches said limitations at para. 46: a charging query may include a request for a route between two locations, where the route may be determined based on compatible charging stations that are distributed between the two locations; para. 55: in a scenario in which a user is moving while the communications of FIG. 6 are being performed, the user location may change over time. Accordingly, the changed parameters referenced at 624 may relate to a changed location of the vehicle, and the identification of additional groups of charging stations may be performed continuously as updates of the changing location are provided to the server. Correspondingly, the generation and transmission of mapping data may be iteratively performed for each charging station group identification, and the map displayed may be continuously updated to reflect the changing location and identified charging stations. Accordingly, as the vehicle travels along a route, the server may iteratively service additional charging station queries automatically, where servicing the additional charging station queries includes, for each additional charging station query, identifying a new subset of charging stations that includes charging stations within a threshold distance of the vehicle and that fit the parameters of the additional charging station query (e.g., match the connector code(s) of the query), and transmitting updated mapping data corresponding to the new subset of the charging stations to the client device. Similar iterative processing may be performed in the course of determining a route for a user, in which the adjusted parameters may include different locations along different candidate routes to identify charging stations along the different candidate routes. Therefore, any route/portions of the searchable data are pruned where there the compatible charging stations are not satisfied; fig. 8: conditions.
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon, Zhou and updated queries of Teske in order to effectively provide a user with convenient vehicle experience with user-friendly visual identity for vehicle charging stations that fit parameters of a prior charging station query that are within a threshold distance of a newly-entered location.
As per claim 2, Cahoon et al. teaches
wherein the first content is of the one or more time intervals, and the updated query includes the one or more values representing a beginning time corresponding to the beginning frame and an end time corresponding to the end frame based at least on the analyzing the first content confirming the one or more relationships are satisfied in the beginning frame and the end frame (col. 11:24-36: a digital video recorder, data, media, audio, video, streaming technology servers, iTV, etc. Thus, managing digital images/sequence of frames in a video; col. 10:43-47: a response of the autonomous controller 506 can represent how an autonomous vehicle is likely to respond in a real environment. Thus, in a real environment, the frames from videos of the autonomous vehicle are analyzed to determine current operational space, conditions, scenarios to effectively control said vehicle; col. 4:29-65: a condition primitive can cause the performance of some action or evaluation of some other condition to be delayed until a condition associated with the condition primitive is satisfied. A "wait" condition can be satisfied upon lapse of the period of time. A "wait for" condition can instruct the simulator not to take any action until another condition is met. That is, a "wait for" condition can be satisfied upon the satisfaction of another condition; fig. 2: sequence start 232, test car in intersection 234/beginning frame, wait 1 second 236, follow observer car 238.)
Cahoon does not explicitly teach first content captured in one or more frames of sequences of frames; from a beginning frame and an end frame.
Zhou et al. teaches
first content captured in one or more frames of sequences of frames stored in searchable data (para. 66-69: video analytics can automatically analyze the video sequences from the cameras and send alarms for events of interest. Video analytics can be trained to recognize certain objects. The video analytics system receives video frames from a video source. The video frames can also be referred to herein as a video picture or a picture. The video frames can be part of one or more video sequences. The video source can include a video capture device (e.g., a video camera, a camera phone, a video phone, or other suitable capture device), a video storage device; para. 130, 236: comparing a size of the object tracker in the current video frame to a size of the object tracker in a last video frame at which object classification was performed for the object tracker).
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon and object/blob tracking to sequences of received data frames of Zhou in order to effectively analyze and identify scenarios in the captured video(s) to alert users of certain events or control objects based on scenarios as taught by Cahoon.
As per claim 3, Cahoon et al. teaches
wherein the converting includes replacing time variables that are defined by the initial query and that indicate periods of time to evaluate whether the one or more relationships are satisfied with explicit values representing a subset of the periods of time to evaluate based at least on the analyzing the first content indicating the one or more relationships are satisfied in the subset of the periods of time (col. 2:26-33: simulations can be used to understand the operational space of an autonomous vehicle (i.e. the envelope of parameters in which the autonomous controller effectively controls the autonomous vehicle) in view of surface conditions, ambient noise, faulty components, etc.; col. 3:6-26: one scenario can be defined as one entity, a car for example, having certain dimensions and velocity, positioned to be a predetermined distance ahead of a simulated test autonomous vehicle, also having specified dimensions and velocity, along a roadway; col. 4:29-65: a condition primitive can cause the performance of some action or evaluation of some other condition to be delayed until a condition associated with the condition primitive is satisfied. A "wait" condition can be satisfied upon lapse of the period of time. A "wait for" condition can instruct the simulator not to take any action until another condition is met; col. 15:16-24: the systems and methods described herein can be implemented using programming languages such as Flash™, JAVA™, ... HTML, etc., or a combination of programming languages, including com piled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment; col. 14:4-5: binary runtime environment for wireless, thus, a Java or other application is converted to be used in BREW or binary converter; col. 18:47-53: the simulation application can assign an iteration variable which holds the assigned parameters for every combination of values. That is, the simulation application can generate permutations for combinations of possible values associated with the primitives. A nonlimiting example of permutations resulting from an outer product is shown above with reference to TABLE 1; col. 22:9-21).
As per claim 4, Cahoon does not explicitly teach claim 4.
Zhou teaches
first content is from a subset of frames of a sequence of frames, and the one or more values exclude the sequence of frames from the one or more time intervals based at least on the analyzing the first content indicating the one or more relationships are not satisfied in at least one frame from the subset of frames (para. 8-9, 64: capturing video sequences of the scene or environment; para. 74: a timestamp can be used; para. 69: the video frames can be part of one or more video sequences/sub-sequences; video analytics can automatically analyze the video sequences from the cameras and send alarms for events of interest; para. 71: object detection and tracking allow the video analytics system to provide various end-to-end features, such as the video analytics features described above. For example, intelligent motion detection, intrusion detection, and other features can directly use the results from object detection and tracking to generate end-to-end events. Other features, such as people, vehicle, or other object counting and classification can be greatly simplified based on the results of object detection and tracking; para. 99: a learned representation may include a quantitative value, a category, a vector, a list of data objects, or the like; para. 192: provide useful information in time-critical scenarios).
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon and Zhou to effectively analyze and evaluate different scenarios in Cahoon teachings in order to perform necessary tasks sufficiently.
As per claim 5, Cahoon et al. teaches
wherein the initial query and the updated query have a same solution space (col. 4:3-56: the SDL/Scenario Description Language enables running an "exhaustive enumerative" search (initial query) over conditions, as well as enabling a user to run tests often as they develop a system. Using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated. The SDL enables running an "exhaustive enumerative" search over conditions, as well as enabling a user to run tests often as they develop a system; col. 5:29-35; col. 18:1-18: instantiating the sequence within the map, simulations can be used to understand the operational space of an autonomous vehicle in view of surface conditions, ambient noise, faulty components, etc. Thus, sequences of operations leading to the same result/solution space.)
As per claim 6, Cahoon does not teach said claim.
Zhou teaches
searchable data includes sequences of frames stored in a data store prior to the receiving of the data representative of the initial query (para. 64-67: a video analytics system can obtain a sequence of video frames from a video source and can process the video sequence to perform a variety of tasks; Video analytics can automatically analyze the video sequences from the cameras and send alarms for events of interest),
a query result of the query results to the initial query includes a sub-interval of a sequence of the sequences, the sub-interval being determined to satisfy the initial query (fig. 7: video analytics system analyzing received video frames. The object tracking system performs Blob tracking and updating system; para. 66: when there are two or more cameras to monitor or as time goes beyond a certain period of time (e.g., 20 minutes), the operator's ability to monitor the video and effectively respond to events; para. 72-75: when the blob tracker is updated in the previous frame (after being associated with the previous blob in the previous frame), updated information for the blob tracker can include the tracking information for the previous frame and also prediction of a location of the blob tracker in the next frame (which is the current frame in this example). The prediction of the location of the blob tracker in the current frame can be based on the location of the blob in the previous frame. A history or motion model can be maintained for a blob tracker, including a history of various states, a history of the velocity, and a history of location, of continuous frames; para. 131: tracking a tracked object/blob when certain conditions are met (e.g., the blob has been tracked for a certain number of frames, a certain period of time, and/or other suitable conditions). The blob tracking and updating system 714 can also include a video analytics manager that can record object detection and tracking events);
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon and object/blob tracking to sequences of received data frames of Zhou in order to effectively analyze and identify scenarios in the captured video(s) to alert users of certain events or control objects based on scenarios as taught by Cahoon – See Cahoon, col. 2:13-67 and Zhou, para. 100: the bounding box of the blob tracker can be the bounding box of a blob associated with the blob tracker in a previous frame. Such techniques can be good for certain controlled scenarios, such as well-aligned lane conveying.
As per claim 7, Cahoon et al. teaches
determining, based at least on one or more of map data or ground truth data corresponding to the sequences of frames, driving surface coordinates corresponding to the one or more objects and the ego-machine (col. 4:3-38: using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated. Coordinate systems can be utilized to describe positions and/or speeds (or velocities) of entities in a scenario. Coordinate systems can include inertial coordinate systems, track based coordinate systems, map based coordinate systems; col. 2:26-31: simulations can be used to understand the operational space of an autonomous vehicle (i.e. tl1e envelope of parameters in which the autonomous controller effectively controls the autonomous vehicle) in view of surface conditions, ambient noise, faulty components, etc.; col. 18:15-18: instantiating the sequence within the map, simulations can be used to understand the operational space of an autonomous vehicle in view of surface conditions),
the converting includes adding one or more sub-expressions to the initial query to limit the first search space to one or more driving surfaces corresponding to the driving surface coordinates (col. 4:3-38: using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated. Coordinate systems can be utilized to describe positions and/or speeds (or velocities) of entities in a scenario. Coordinate systems can include inertial coordinate systems, track based coordinate systems, map based coordinate systems; col. 2:26-31: simulations can be used to understand the operational space of an autonomous vehicle (i.e. the envelope of parameters in which the autonomous controller effectively controls the autonomous vehicle) in view of surface conditions, ambient noise, faulty components, etc.; col. 18:15-18: instantiating the sequence within the map, simulations can be used to understand the operational space of an autonomous vehicle in view of surface conditions, ambient noise, faulty components, etc.);
As per claims 8, 23, Cahoon et al. teaches
wherein the converting the initial query includes: determining, using a compiler and based at least in part on one or more rules, an relationship of the one or more relationships is more likely to fail; based at least on the determining, ordering, using the compiler, the logical expression such that the relationship is searched for before another of the one or more relationships that is less likely to fail (col. 4:3-56: using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated; col. 7:8-59: the scenario description language allows the user to specify how certain primitives are associated with one another, for example, using a linear temporal logic. In one example, a user defines an association between primitives by specifying a sequence. A sequence can include multiple steps, which are to be performed in sequential order by the simulator; fig. 7; col. 14:4-5: binary runtime environment for wireless, thus, a Java or other application is converted to be used in BREW or binary converter; col. 15:15-25: the systems and methods described herein can be implemented using programming languages such as Flash™, JAVA™, … HTML, etc., or a combination of programming languages, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment).
Zhou also teaches at para. 67-69: video analytics can also perform video search (e.g., extracting basic activity for a given region) and video summary (e.g., extraction of the key movements). In some instances, event detection can be performed by video analytics, including detection of fire, smoke, fighting, crowd formation, or any other suitable even the video analytics is programmed to or learns to detect. A detector can trigger the detection of an event of interest and can send an alert or alarm to a central control room to alert a user of the event of interest; para. 71: object detection and tracking allow the video analytics system to provide various end-to-end features, such as the video analytics features described above. For example, intelligent motion detection, intrusion detection, and other features can directly use the results from object detection and tracking to generate end-to-end events. Other features, such as people, vehicle, or other object counting and classification can be greatly simplified based on the results of object detection and tracking; col. 151-154: the classification task management engine can be applied to select N requests once every M frames, where N is an integer greater than 1; para. 352-355: where components are described as being “configured to” perform certain operations.
As per claim 9, Cahoon et al. teaches
determining, based at least on one or more of map data or ground truth data corresponding to the sequences of frames, a threshold spatial distance from the ego-machine, and the converting includes adding one or more sub-expressions to the initial query to limit the second search space to locations that are within the threshold spatial distance from the ego-machine (col. 3:21-26: spatial distance; col. 4:3-65: using the SDL or Scenario Description Language, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated; coordinate systems can be utilized to describe positions and/or speeds of entities in a scenario. Coordinate systems can include inertial coordinate systems, track based coordinate systems, map-based coordinate systems; col. 2:13-67: autonomous vehicles that simulations can be used to query, analyze, test a simulated environments or machine learning in an operational space of real world environments; col. 5:1-34: a "distance between or near condition" can be satisfied when a distance between specified entities is determined to be within a threshold distance. In at least one example, when a distance between specified entities is within a threshold distance, a Boolean signaling can be relayed to the simulator indicating that the distance between two entities is below some user specified value. Thus, adding expression/sub-expressions to the initial query/search to the one or more objects that are within a threshold distance).
Cahoon does not explicitly teach sequences of frames.
Zhou teaches at para. 64-67: capturing video sequences of the scene or environment. Video analytics provides variety of tasks ranging a from immediate detection of events of interest, to analysis of pre-recorded video for the purpose of extracting events in a long period of time, as well as many other tasks. Video analytics can automatically analyze the video sequences from the cameras and send alarms for events of interest. Video analytics can operate as an Intelligent Video Motion Detector by detecting moving objects and by tracking moving objects; para. 69: the video frames can be part of one or more video sequences/sub-sequences; para. 67-69: video analytics can also perform video search (e.g., extracting basic activity for a given region) and video summary (e.g., extraction of the key movements). In some instances, event detection can be performed by video analytics, including detection of fire, smoke, fighting, crowd formation, or any other suitable even the video analytics is programmed to or learns to detect. A detector can trigger the detection of an event of interest and can send an alert or alarm to a central control room to alert a user of the event of interest; para. 71: object detection and tracking allow the video analytics system to provide various end-to-end features, such as the video analytics features described above. For example, intelligent motion detection, intrusion detection, and other features can directly use the results from object detection and tracking to generate end-to-end events. Other features, such as people, vehicle, or other object counting and classification can be greatly simplified based on the results of object detection and tracking.
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon and Zhou in order to effectively analyze the scenarios in the captured video(s) and including distances between objects to effectively manage the autonomous vehicle – See para. 98-100.
As per claims 10, 17, Cahoon et al. teaches
wherein the system is comprised in at least one of: a control system for an autonomous or semi-autonomous machine; a perception system for an autonomous or semi-autonomous machine; a system for performing simulation operations; a system for performing deep learning operations; a system implemented using an edge device; a system implemented using a robot; Page 82 of 88Non-provisional ApplicationSHB Matter No.: 41651.352195 NVIDIA Matter No.: 20-SC-0426US01a system incorporating one or more virtual machines (VMs); a system implemented at least partially in a data center; or a system implemented at least partially using cloud computing resources (col. 2:19-67: simulations can be used to validate software (i.e., an autonomous controller) being run on autonomous vehicles to ensure that the software is able to safely control such autonomous vehicles; determining an amount of redundancy that is required in an autonomous controller, or how to modify a behavior of the autonomous controller based on what is learned through simulations; col. 10:5-57: deep learning operations, virtual systems; col. 15:51-57).
As per claim 13, Cahoon et al. teaches
wherein the one or more relationships are defined as alternatives such that the analyzing the second content includes identifying at least a first scenario corresponding to a first alternative and a second scenario corresponding to a second alternative (col. 2:65-67: the domain specific language can be used to determine a limited number (subset) of scenarios to simulate which will provide useful information; col. 4:43-65: additionally and/or alternatively, an "inversion modifier" condition can invert a condition. Conditions can also include Boolean operators, such as, but not limited to, and, or not, xor, nor, nand, etc. such that multiple conditions as primitives can be combined; col. 18:63-67: multiple scenarios can be generated based at least in part on the simulation application instantiating the sequence within a map in view of the various permutations of possible values. Each iteration can then be associated with a more specific scenario).
As per claim 14, Cahoon et al. teaches
wherein the converting includes updating a search ordering to an updated search ordering such that one or more conditions more likely to fail are ordered before one or more conditions less likely to fail (col. 2:56-67; col. 4:7-12: the SDL enables running an "exhaustive enumerative" search over conditions, as well as enabling a user to run tests often as they develop a system. Additionally, memory and processing requirements are reduced by limiting iterations by iterating over scenarios which provide useful information; col. 6:41-53: faults can represent failures associated with an entity and/or environment. For instance, as non-limiting examples, a fault can represent a failure of a sensor associated with an entity, a failure of a tire associated with an entity, or a failure of another component associated with an entity. Fault primitives can indicate to the simulator to cause a respective fault in the simulated environment; col. 7:8-13; col. 10:18-47; col. 16:37-49).
As per claim 16, Cahoon et al. teaches
wherein based at least on determining the at least one second sub-sequence of frames does not satisfy the one or more relationships (col. 2:13-67: techniques for creating a domain specific language for use in constructing simulations. Simulations can be used to validate software (i.e., an autonomous controller) being run on autonomous vehicles to ensure that the software is able to safely control such autonomous vehicles. For instance, simulations can be used to understand the operational space of an autonomous vehicle (i.e. the envelope of parameters in which the autonomous controller effectively controls the autonomous vehicle) in view of surface conditions, ambient noise, faulty components, etc. (world states of captured scenarios); col. 4:43-65: a condition primitive can cause the performance of some action or evaluation of some other condition to be delayed until a condition associated with the condition primitive is satisfied. Condition primitives can include, but are not limited to, a "wait" condition, a "wait for" condition, a "distance between or near" condition, a "speed" condition, an "in region" condition, etc.; a "wait" condition can be satisfied upon lapse of the period of time. A "wait for" condition can instruct the simulator not to take any action until another condition is met. That is, a "wait for" condition can be satisfied upon the satisfaction of another condition; col. 5:23-25: an "in region" condition can instruct the simulator to delay the performance of some action or evaluation of some other condition until an entity is within a specified region.)
Cahoon does not explicitly teach at least one second sub-sequence of frames from the at least one sequence of frames is not identified by the query results as capturing the scenario.
Zhou teaches
at least one second sub-sequence of frames from the at least one sequence of frames is not identified by the query results as capturing the scenario (para. 15: a tracker is assigned the lost state when an object for which the tracker was associated with in a previous video frame is not detected in subsequent video frame; para. 64: capturing video sequences of the scene or environment; para. 66-67: video analytics can automatically analyze the video sequences from the cameras and send alarms for events of interest; video analytics can also perform video search (e.g., extracting basic activity for a given region) and video summary (e.g., extraction of the key movements). In some instances, event detection can be performed by video analytics, including detection of fire, smoke, fighting, crowd formation, or any other suitable even the video analytics is programmed to or learns to detect. A detector can trigger the detection of an event of interest and can send an alert or alarm to a central control room to alert a user of the event of interest; the video frames can be part of one or more video sequences/sub-sequences). Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon and Zhou in order to effectively manage the storing, mapping and facilitating searching sub-sequences of frames and to filter out different screnario(s).
As per claim 19, Cahoon et al. teaches
wherein the domain includes one or more of an autonomous machine domain, an
autonomous driving domain, a semi-autonomous driving domain, or a robotics
domain (col. 2:13-67: techniques for creating a domain specific language for use in
constructing simulations. Simulations can be used to validate software (i.e., an
autonomous controller) being run on autonomous vehicles to ensure that the software is
able to safely control such autonomous vehicles. Simulations can be useful to inform
the hardware design of autonomous vehicles, such as optimizing placement of sensors
on an autonomous vehicle.)
As per claim 20, Cahoon et al. teaches
evaluating, in the query results, one or more speed or acceleration conditions of the ego-machine during the scenario or one or more distance-based conditions during the scenario (col. 2:26-33: simulations can be used to understand the operational space of an autonomous vehicle (i.e., the envelope of parameters in which the autonomous controller effectively controls the autonomous vehicle) in view of surface conditions, ambient noise, faulty components, etc.; col. 3:6-26: one scenario
can be defined as one entity, a car for example, having certain dimensions and velocity, positioned to be a predetermined distance ahead of a simulated test autonomous vehicle, also having specified dimensions and velocity, along a roadway; col. 4:7-65: coordinate systems can be utilized to describe positions and/or speeds (or velocities) of entities in a scenario. Coordinate systems can include inertial coordinate systems, track based coordinate systems, map-based coordinate systems. Condition primitives can include, but are not limited to, a "wait" condition, a "wait for" condition, a "distance between or near" condition, a "speed" condition, an "in region" condition, etc.; col. 5:1-33).
As per claim 22, Cahoon et al. teaches
the sequences of frames include real-world video of an environment (col. 1:32-39; col. 9:35-41: figs. 4A-4E, multiple scenarios can be created by a single outer product, all categorized by having a single T-intersection and a single test car. The scenario represented by fig. 4A includes the test car in a first starting position. There are two pedestrians 402A and 4028 present and no other cars present; col. 10:42-47: a response of the autonomous controller can represent how an autonomous vehicle is likely to respond in a real environment).
Zhou also teaches at para. 66: various research studies and real-life experiences indicate that in a surveillance system, for example, a human operator typically cannot remain alert and attentive for more than minutes, even when monitoring the pictures from one camera. When there are two or more cameras to monitor or as time goes beyond a certain period of time (e.g., 20 minutes), the operator's ability to monitor the video and effectively respond to events is significantly compromised. Video analytics can automatically analyze the video sequences from the cameras and send alarms for events of interestPage 81 of 88Non-provisional ApplicationSHB Matter No.: 41651.352195.
As per claim 24, Cahoon et al. teaches
wherein, once a condition associated with an relationship of the one or more search relationships is determined to have failed during the analyzing, determining a subsequent time in the video/data where the explicit relationship no longer fails, and restarting the analyzing at the subsequent time such that the analyzing the second content does not include searching a subset of the searchable data corresponding to the video/data between the time when the condition failed and the subsequent time (col. 4:8-65: the SDL/Scenario Description Language enables running an "exhaustive enumerative" search over conditions/initial query over conditions, as well as enabling a user to run tests often as they develop a system. A "wait" condition can instruct the simulator to delay the performance of some action or evaluation of some other condition a specified period of time before proceeding; col. 5, line 1-col. 6, line 30: when a distance between specified entities is within a threshold distance, a Boolean signaling can be relayed to the simulator indicating that the distance between two entities is below some user specified value; col. 8:3-9: a sequence instructs the simulator to first wait for the test car to approach the intersection, wait 1 second and then have the test car follow the observer car. Thus, the initial query is to search over conditions, e.g., if “approach the intersection”, when the condition is not satisfied, search for the current/subsequence video sequence/time from the last fail to see if said condition is satisfied. If no longer fails, search for next condition and thus, next action: a time interval “1 second” etc.; col. 9:24-col.10:23: consider an outer product where a user specifies a range of starting positions and a range of a number of other primitives (e.g., entities with associated dimensions, velocities, and categories) to be instantiated … limiting which scenarios are created for validation and testing can reduce a number of required computations. That is, limiting which scenarios are created for validation and testing can reduce a number of required computations that are performed for scenarios that are likely to output useful information; col. 11:24-36: a digital video recorder, data, media, audio, video, streaming technology servers, iTV, etc. Thus, managing digital images/sequence of frames in a video).
Cahoon does not explicitly teach sequences of frames.
Zhou teaches at para. 64: capturing video sequences of the scene or environment; para. 69: the video frames can be part of one or more video sequences/sub-sequences; para. 67-69: video analytics can also perform video search (e.g., extracting basic activity for a given region) and video summary (e.g., extraction of the key movements). In some instances, event detection can be performed by video
analytics, including detection of fire, smoke, fighting, crowd formation, or any other suitable even the video analytics is programmed to or learns to detect. A detector can trigger the detection of an event of interest and can send an alert or alarm to a central control room to alert a user of the event of interest; para. 71: object detection and tracking allow the video analytics system to provide various end-to-end features, such as the video analytics features described above. For example, intelligent motion
detection, intrusion detection, and other features can directly use the results from object detection and tracking to generate end-to-end events. Other features, such as people, vehicle, or other object counting and classification can be greatly simplified based on the results of object detection and tracking.
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings or Cahoon and Zhou in order to effectively analyze the scenarios in the captured video(s) to alert users of certain events - See para. 67-69.
As per claim 25, Cahoon et al. teaches
wherein the analyzing the first content includes determining that the one or more relationships are satisfied by the second content at a beginning time and an end time of an interval corresponding to the at least one sequence of video (col. 2:13-67: autonomous vehicles that simulations can be used to query, analyze, test a simulated environments or machine learning in an operational space of real world environments. Thus, continuously learning by querying current conditions of the current operational space and time to accurately produce scenarios which encountered by an autonomous vehicle to ensure that the software is able to safely control such autonomous vehicles; col. 4:8-65: the SDL/Scenario Description Language enables running an "exhaustive enumerative" search over conditions/initial query over conditions, as well as enabling a user to run tests often as they develop a system. A "wait" condition can instruct the simulator to delay the performance of some action or evaluation of some other condition a specified period of time before proceeding; col. 5, line 1-col. 6, line 30: when a distance between specified entities is within a threshold distance, a Boolean signaling can be relayed to the simulator indicating that the distance between two entities is below some user specified value; col. 8:3-9: a sequence instructs the simulator to first wait for the test car to approach the intersection, wait 1 second and then have the test car follow the observer car. Thus, the initial query is to search over conditions, e.g., if not “approach the intersection” then the condition is not satisfied, search for the current/subsequence video sequence/time from the last fail to see if said condition is satisfied. If no longer fails, search in next/second sequence of video for next condition and thus, continue to the next action: a time interval “1 second” etc.); col. 9:24-col.10:47: consider an outer product where a user specifies a range of starting positions and a range of a number of other primitives (e.g., entities with associated dimensions, velocities, and categories) to be instantiated… limiting which scenarios are created for validation and testing can reduce a number of required computations. For example, a machine learning mechanism can build, modify, or otherwise utilize a data model that is created from example inputs and makes predictions or decisions using the data model. A response of the autonomous controller can represent how an autonomous vehicle is likely to respond in a real environment;
col. 11:24-36: a digital video recorder, data, media, audio, video, streaming technology servers, iTV, etc. Thus, search for scenario(s) in certain time or space between one or more objects are based on received data relating to the object even of a simulated environment data.
Cahoon does not explicitly teach frames.
Zhou teaches
at least one sequence of frames (para. 15: a tracker is assigned the lost state when an object for which the tracker was associated with in a previous video frame is not detected in subsequent video frame; para. 64: capturing video sequences of the scene or environment; para. 66-67: video analytics can automatically analyze the video sequences from the cameras and send alarms for events of interest; video analytics can also perform video search (e.g., extracting basic activity for a given region) and video summary (e.g., extraction of the key movements). In some instances, event detection can be performed by video analytics, including detection of fire, smoke, fighting, crowd formation, or any other suitable even the video analytics is programmed to or learns to detect. A detector can trigger the detection of an event of interest and can send an alert or alarm to a central control room to alert a user of the event of interest; the video frames can be part of one or more video sequences/sub-sequences).
Zhou also teaches at para. 69: the video frames can be part of one or more video sequences/sub-sequences; video analytics can automatically analyze the video sequences from the cameras and send alarms for events of interest; para. 131: tracking a tracked object/blob when certain conditions are met (e.g., the blob has been tracked for a certain number of frames, a certain period of time, and/or other suitable conditions); col. 151-154: the classification task management engine can be applied to select N requests once every M frames, where N is an integer greater than 1; para. 352-355: where components are described as being “configured to” perform certain operations).
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon and Zhou in order to effectively manage the storing and facilitating searching/determine sequence/sub-sequences of frames and to identify scenario(s).
Claim(s) 12, 15, 21 are rejected under 35 U.S.C. 103 as being unpatentable over Cahoon et al. (US 10489529) in view of Zhou (US 20190130188) and further in view of Teske (US 20190383637) and Tomkins (20210295822).
As per claim 12, Cahoon et al. teaches
wherein the declarative language query includes a first portion corresponding to a domain associated with the one or more actors and a second portion that is independent of the domain (figs. 1-2: test car, observer car / actor etc.; col. 3:37-67; col. 10:1-47: the machine learning mechanism can leverage data associated with which scenarios pass (i.e., succeed) or fail (i.e., do not succeed) (per predefined criteria) and can determine which sets of scenarios collectively pass or fail. Based on said determination, the machine learning mechanism can determine that scenarios in a set of scenarios are redundant. The data model can be trained using supervised learning algorithms etc.; a response of the autonomous controller can represent how an autonomous vehicle is likely to respond in a real environment.)
Cahoon, Zhou, Teske do not explicitly teach a world model or ontology.
Tomkins teaches in fig. 2: ontology data repository with ontology models a logical and physical architecture of data stored in an ontology model; para. 53: different data types may be combined to update an ontology data model, such as one stored in an ontology data model record; the ontology data model record may store values for record fields such as object categories, relationships between the categories, directional indicators of the relationships, or the like; para. 124-125: a trained learning system, create a structured knowledge base of a knowledge fabric usable to provide data in response to queries; para. 192: provide useful information in time-critical scenarios. Additionally, summarizations provide the practical benefit of reducing cognitive load on users during a search operation through natural-language text by providing users with relevant information that helps them determine which documents to analyze).
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon, Zhou and Tomkins in order to effectively manage the storing, mapping and facilitating domain knowledge sharing more accurately based on documents/scenarios repetitiveness - See para. 313.
As per claim 15, Cahoon teaches structured data: col. 17:50-64. Zhou teaches at para. 62-66: data structures; para. 67-69: video analytics can also perform video search (e.g., extracting basic activity for a given region) and video summary (e.g., extraction of the key movements). In some instances, event detection can be performed by video analytics, including detection of fire, smoke, fighting, crowd formation, or any other suitable even the video analytics is programmed to or learns to detect. A detector can trigger the detection of an event of interest and can send an alert or alarm to a central control room to alert a user of the event of interest; para. 71: object detection and tracking allow the video analytics system to provide various end-to-end features, such as the video analytics features described above. For example, intelligent motion detection, intrusion detection, and other features can directly use the results from object detection and tracking to generate end-to-end events. Other features, such as people, vehicle, or other object counting and classification can be greatly simplified based on the results of object detection and tracking.
Cahoon, Zhou, Teske do not explicitly teach caches.
Tomkins et al. teaches
wherein, during the analyzing, a subset of the searchable data is converted to structured data and stored in one or more caches such that at least a portion of the analyzing is performed using the structured data (claim 1: transforming unstructured natural-language text into structured data, comprising: obtaining, with a computer system, a corpus of natural-language text documents; classifying, with the computer system, the natural language text documents into a plurality of classes corresponding to different fields of domain knowledge; para. 192: provide useful information in time-critical scenarios. Additionally, summarizations provide the practical benefit of reducing cognitive load on users during a search operation through natural-language text by providing users with relevant information that helps them determine which documents to analyze; para. 211: a self-balanced search tree, prefix tree, or other index may be loaded into a cache memory to increase data retrieval speeds, where a cache memory may include an L1 cache, L2 cache, L3 cache, or another cache memory of a different or mixed cache level. A cache memory may refer to a hardware cache that is integrated with a computer processor and characterized by being faster to access than other memory of a computer system and may include one or more SRAM components. By allocating ontology-specific indices into a cache memory of a computing device, some embodiments may accelerate the speed by which ontology specific text summarization is performed.)
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon, Zhou, Teske and Tomkins in order to effectively manage the storing, mapping and facilitating domain knowledge sharing in faster storage.
As per claim 21, Cahoon et al. teaches
wherein the analyzing includes performing multiple iterations of searching through the searchable data, and the operations further comprise: during each iteration of the multiple iterations, storing values associated with the iteration in the one or more caches, wherein subsequent iterations after the iteration include searching the values stored in the one or more caches (col. 4:3-56: using the SDL, it is possible to have concise definitions of test scenarios, which allows computer code to be highly readable and allows for quick updates of a large suite of test scenarios whenever map data is updated. The SDL enables running an "exhaustive enumerative" search over conditions, as well as enabling a user to run tests often as they develop a system. Memory and processing requirements are reduced by limiting iterations by iterating over scenarios which provide useful information; col. 8:19-67: each iteration can be associated with a more specific scenario and each scenario can be output for use in a simulation for testing and validation; col. 18:63-67: multiple scenarios can be generated based at least in part on the simulation application instantiating the sequence within a map in view of the various permutations of possible values. Each iteration can then be associated with a more specific scenario; fig. 5: memory used by the simulation application and autonomous controller).
Cahoon teaches memory. Cahoon, Zhou, Teske do not explicitly teach cache memory.
Tomkins teaches one or more caches at para. 276: a cache memory may include an L1 cache, L2 cache, L3 cache, where different types of cache memory systems may indicate different levels of available memory or the speed of memory access. As described elsewhere in this disclosure, some embodiments may load one or more elements of an index into cache memory in response to a determination that a user having a user context parameter associated with the index is using a computing device to perform one or more of the operations.
Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon, Zhou, Teske and Tomkins in order to effectively manage the storing and retrieving of searching data quickly based on the repetitiveness or reuse of processing data elements – See Tomkins, para. 317.
Claim(s) 26 is/are rejected under 35 U.S.C. 103 as being unpatentable over Cahoon et al. (US 10489529) in view of Zhou (US 20190130188) and further in view of Teske (US 20190383637) and Raichle et al. (US 20150214858).
As per claim 26, Cahoon et al. teaches
wherein the analyzing the second content at col. 4:8-65: the SDL/Scenario Description Language enables running an "exhaustive enumerative" search over conditions/initial query over conditions, as well as enabling a user to run tests often as they develop a system. A "wait" condition can instruct the simulator to delay the performance of some action or evaluation of some other condition a specified period of time before proceeding; col. 5, line 1-col. 6, line 30: when a distance between specified entities is within a threshold distance, a Boolean signaling can be relayed to the simulator indicating that the distance between two entities is below some user specified value; col. 8:3-9: a sequence instructs the simulator to first wait for the test car to approach the intersection, wait 1 second and then have the test car follow the observer car. Thus, the initial query is to search over conditions, e.g., if not “approach the intersection” then the condition is not satisfied, search for the current/subsequence video sequence/time from the last fail to see if said condition is satisfied. If no longer fails, search in next/second sequence of video for next condition and thus, continue to the next action: a time interval “1 second” etc.
Cahoon, Zhou, Teske do not teach includes executing one or more of a short-circuit evaluation algorithm, a minimal evaluation algorithm, or a McCarthy evaluation algorithm.
Raichle et al. teaches
wherein the analyzing the second content includes executing one or more of a short-circuit evaluation algorithm, a minimal evaluation algorithm, or a McCarthy evaluation algorithm (para. 10: a control device which is coupled to the evaluation device and which is designed to switch the inverter from a short-circuit state to a freewheeling mode as a function of the detected input voltage; para. 20). Thus, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Cahoon, Zhou, Teske and Raichle et al. in order to effectively evaluate different scenarios in Cahoon teachings.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kattepur et al. (US 20210049037) teaches at para. 53: FIG. 7 which covers the scenario of the robot action of grabbing an item from the shelf in the warehouse failing due to the door and a cup-board remaining in closed state.
Turcot (US 20210125065) teaches at para. 47: the pruning can also be used to reduce computational complexity by reducing a number of computations, by shrinking a search space, etc.
Bronstein et al. teaches content captured in one or more frames of sequences of frames stored in (para. 6: automatically matching high-level and human understandable semantic description to lower level video content that may be automatically handled by one or more processors belongs to the general category of pattern recognition problems, usually referred to as video search; para. 30: analyzing input media content and automatically generating descriptors that are associated with the image and/or audio contents of various time portions of the input media content.
Mao (US 20210365707) teaches at para. 6: determine a region of interest in a first frame of a sequence of frames, the region of interest in the first frame including an object having a size in the first frame.
Dembo (US 20190197206) teaches at para. 195-196: systematize the generation of scenarios so as to enable them to be generated automatically. Machine learning unit 120 processes input data to detect events and outcomes (e.g., shocks) that trigger the forward looking scenario analysis.
Unnikrishnan (US 20210406262) teaches at para. 42-47: the scenario search module 202 can be configured to communicate and operate with the at least one data store 220. The at least one data store 220 can be configured to maintain and store various types of data. For example, the data store 220 can store information describing a variety of scenarios.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LINH BLACK whose telephone number is (571)272-4106. The examiner can normally be reached 9AM-5PM EST M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tony Mahmoudi can be reached on 571-272-4078. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/LINH BLACK/Examiner, Art Unit 2163 6/4/2026
/TONY MAHMOUDI/Supervisory Patent Examiner, Art Unit 2163
Prosecution Insights