DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Status of Claims
Claims 29-57 are pending and have been examined below.
Response to Arguments
Applicant's arguments with respect to 35 USC 112a have been considered and are persuasive. The rejections are withdrawn.
Applicant’s arguments and amendments with respect to 35 USC 102 and 103 have been considered but are moot because the arguments do not apply to any of the references being used in the current rejection.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 USC 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 29, 30, 36-44, 50-53, 56 and 57 are rejected under 35 USC 103 as being unpatentable over US20170008521 (“Braunstein”) in view of US20170168485 (“Berntorp”).
Claim 29
Braunstein discloses a navigation system for a host vehicle (0003), the navigation system comprising: at least one processor comprising circuitry and a memory (0051), wherein the memory includes instructions that when executed by the circuitry cause the at least one processor to:
receive a plurality of images acquired by a camera, the plurality of images being representative of an environment of the host vehicle (0052 a method of navigating an autonomous vehicle may include receiving, from an image capture device, at least one image representative of an environment of the vehicle;);
analyze at least one of the plurality of images to identify navigational state information associated with the host vehicle (0052 analyzing, using at least one processor, the at least one image to identify two or more landmarks located in the environment of the vehicle; determining, for each of the two or more landmarks, a directional indicator relative to the vehicle; determining a current location of the vehicle relative to the road junction based on an intersection of the directional indicators for the two or more landmarks;);
determine a plurality of first potential navigational actions for the host vehicle based on the navigational state information (0605 processing unit 110 may determine the autonomous steering action based on a kinematic and physical model of the vehicle, which may include the effects of a variety of possible autonomous steering actions on the vehicle or on a user of vehicle 200. Processing unit 110 may implement a selection criteria for selecting at least one autonomous steering action from the plurality of autonomous steering actions. In other exemplary embodiments, processing unit 110 may determine an autonomous steering action based on a “look ahead” operation, which may evaluate portions of road segment 3400 located in front of current location 3418 of vehicle 200.);
determine respective future navigational states for the plurality of first potential navigational actions (0605 processing unit 110 may determine the autonomous steering action based on a kinematic and physical model of the vehicle, which may include the effects of a variety of possible autonomous steering actions on the vehicle or on a user of vehicle 200. Processing unit 110 may implement a selection criteria for selecting at least one autonomous steering action from the plurality of autonomous steering actions. In other exemplary embodiments, processing unit 110 may determine an autonomous steering action based on a “look ahead” operation, which may evaluate portions of road segment 3400 located in front of current location 3418 of vehicle 200. Processing unit 110 may determine an effect of one or more autonomous steering actions on the behavior of vehicle 200 or on a user of vehicle 200 at a location in front of current location 3418, which may be caused by the one or more autonomous steering actions... Thus, for example, processing unit 110 may initially determine an autonomous steering action that may help ensure that heading direction 3430 of vehicle 200 may be aligned with direction 3420 of predetermined road model trajectory 3410 at current location 3418. When processing unit 110 determines that the determined autonomous steering does not comply with one or more constraints imposed by the additional considerations, processing unit 110 may modify the autonomous steering action to help ensure that all the constraints may be satisfied.);
determine a plurality of second potential navigational actions for the host vehicle based on the determined respective future navigational states (0605 processing unit 110 may determine the autonomous steering action based on a kinematic and physical model of the vehicle, which may include the effects of a variety of possible autonomous steering actions on the vehicle or on a user of vehicle 200. Processing unit 110 may implement a selection criteria for selecting at least one autonomous steering action from the plurality of autonomous steering actions. In other exemplary embodiments, processing unit 110 may determine an autonomous steering action based on a “look ahead” operation, which may evaluate portions of road segment 3400 located in front of current location 3418 of vehicle 200. Processing unit 110 may determine an effect of one or more autonomous steering actions on the behavior of vehicle 200 or on a user of vehicle 200 at a location in front of current location 3418, which may be caused by the one or more autonomous steering actions... Thus, for example, processing unit 110 may initially determine an autonomous steering action that may help ensure that heading direction 3430 of vehicle 200 may be aligned with direction 3420 of predetermined road model trajectory 3410 at current location 3418. When processing unit 110 determines that the determined autonomous steering does not comply with one or more constraints imposed by the additional considerations, processing unit 110 may modify the autonomous steering action to help ensure that all the constraints may be satisfied.);
select, based on the plurality of second potential navigational actions, one of the plurality of first potential navigational actions (0605 In yet other exemplary embodiments, processing unit 110 may further account for the presence and behavior of one or more other vehicles in the vicinity of vehicle 200 and a possible (estimated) effect of one or more autonomous steering actions on such one or more other vehicles. Processing unit 110 may implement the additional considerations as overrides. Thus, for example, processing unit 110 may initially determine an autonomous steering action that may help ensure that heading direction 3430 of vehicle 200 may be aligned with direction 3420 of predetermined road model trajectory 3410 at current location 3418. When processing unit 110 determines that the determined autonomous steering does not comply with one or more constraints imposed by the additional considerations, processing unit 110 may modify the autonomous steering action to help ensure that all the constraints may be satisfied... Thus, for example, processing unit 110 may initially determine an autonomous steering action that may help ensure that heading direction 3430 of vehicle 200 may be aligned with direction 3420 of predetermined road model trajectory 3410 at current location 3418. When processing unit 110 determines that the determined autonomous steering does not comply with one or more constraints imposed by the additional considerations, processing unit 110 may modify the autonomous steering action to help ensure that all the constraints may be satisfied.); and
cause at least one adjustment of a navigational actuator of the host vehicle to implement the selected one of the plurality of first potential navigational actions (0605 In yet other exemplary embodiments, processing unit 110 may further account for the presence and behavior of one or more other vehicles in the vicinity of vehicle 200 and a possible (estimated) effect of one or more autonomous steering actions on such one or more other vehicles. Processing unit 110 may implement the additional considerations as overrides. Thus, for example, processing unit 110 may initially determine an autonomous steering action that may help ensure that heading direction 3430 of vehicle 200 may be aligned with direction 3420 of predetermined road model trajectory 3410 at current location 3418. When processing unit 110 determines that the determined autonomous steering does not comply with one or more constraints imposed by the additional considerations, processing unit 110 may modify the autonomous steering action to help ensure that all the constraints may be satisfied.).
Braunstein fails to explicitly disclose: wherein the respective future navigational states are based on a predicted action of at least one vehicle other than the host vehicle. However, Braunstein does disclose controlling the host vehicle based on another vehicle (0260 other obstacles (e.g., other vehicles, pedestrians, debris, etc.)., 0834). Furthermore, Berntorp teaches a system of controlling a vehicle based on a plurality of possible trajectories (abstract), including:
wherein the respective future navigational states are based on a predicted action of at least one vehicle other than the host vehicle (0044 The memory 180 also stores 183 the future motions computed up to the current time and the internal information of the motion planner, including, but not limited to, cost function, values of each computed state, already visited but rejected states, the motion leading up to each state, information about deviations from the desired location of the vehicle, and future predicted motions of obstacles., 0041 An obstacle can be another vehicle or a pedestrian, or a virtual obstacle representing illegal driving behavior, such as the line delimiting the allowed driving lane, a stop line, a yield sign, or from inputs from a driver or passenger., Fig. 3, abstract, 0048, 0051, 0009 For example, the transition to a location from a first location can be more optimal than a transition to the location from a second location, even if the second location is closer to the location that the first location. For example, the heading of the vehicle at the second location can result in a complicated path that reaches the location., 0040).
Braunstein and Berntorp both disclose systems of controlling a vehicle based on a plurality of candidate trajectories and another vehicle. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention and with a reasonable expectation of success to apply the known element(s) of Berntorp to the known system of Braunstein, the latter having been ready for improvement. The combination would have done no more than yield the predictable results of wherein the respective future navigational states are based on a predicted action of at least one vehicle other than the host vehicle.
Claim 30
Braunstein discloses:
wherein the navigational actuator includes at least one of a steering mechanism, a brake, or an accelerator (0605 steering actions).
Claim 36
Braunstein discloses:
wherein at least one of the plurality of first potential navigational actions and the plurality of second potential navigational actions includes merging in front of or behind a detected vehicle (0952 For example, given the permanent nature of an ending lane accompanied by a lane shift, the server may decide it is necessary to change or update the road model. Accordingly, the sever may modify the road model in order to steer or turn to merge at these distances c.sub.1, c.sub.2, d.sub.2, w.sub.1, and w.sub.2 upon approaching lane constraint 7924.)
Claim 37
Braunstein discloses:
wherein the navigational state information includes an indicator of a detected object in the environment of the host vehicle (0970 Determination of whether a particular road condition constitutes a transient condition may be fully automated and performed by one or more server-based systems. For example, in some embodiments, the one or more server based systems may employ automated image analysis techniques based on one or more images captured by cameras onboard a host vehicle. In some embodiments, the image analysis techniques may include machine learning systems trained to recognize certain shapes, road features, and/or objects. For example, the server may be trained to recognized in an image or image stream the presence of a concrete barrier (possibly indicating the presence of a non-transient construction or lane separation condition)).
Claim 38
Braunstein discloses:
wherein the navigational state information includes an indicator of one or more target vehicles detected in the environment of the host vehicle (0023 the one or more sensors may include a speed sensor. The one or more sensors may include an accelerometer. The one or more sensors may include the camera. The at least one navigational constraint may include at least one of a barrier, an object, a lane marking, a sign, or another vehicle. The camera may be included in the vehicle.).
Claim 39
Braunstein discloses:
wherein the navigational state information includes an indicator of one or more lane markings detected in the environment of the host vehicle (0023 the one or more sensors may include a speed sensor. The one or more sensors may include an accelerometer. The one or more sensors may include the camera. The at least one navigational constraint may include at least one of a barrier, an object, a lane marking, a sign, or another vehicle. The camera may be included in the vehicle.).
Claim 40
Braunstein discloses:
wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to determine whether each of the plurality of first potential navigational actions is in compliance with one or more predetermined navigational safety constraints (0605 When processing unit 110 determines that the determined autonomous steering does not comply with one or more constraints imposed by the additional considerations, processing unit 110 may modify the autonomous steering action to help ensure that all the constraints may be satisfied.).
Claim 41
Braunstein discloses:
wherein selecting the one of the plurality of first potential navigational actions includes eliminating at least one noncompliant navigational action of the plurality of first potential navigational actions based on a determination the noncompliant navigational action is not in compliance with one or more predetermined navigational safety constraints (0605 When processing unit 110 determines that the determined autonomous steering does not comply with one or more constraints imposed by the additional considerations, processing unit 110 may modify the autonomous steering action to help ensure that all the constraints may be satisfied.).
Claim 42
Braunstein discloses:
wherein determining the plurality of first potential navigational actions for the host vehicle is further based on the determination whether each of the plurality of first potential navigational actions is in compliance with the one or more predetermined navigational safety constraints (0605 When processing unit 110 determines that the determined autonomous steering does not comply with one or more constraints imposed by the additional considerations, processing unit 110 may modify the autonomous steering action to help ensure that all the constraints may be satisfied.).
Claim 43
Braunstein discloses:
wherein the one or more predetermined safety constraints includes at least one of a pedestrian envelope or a target vehicle envelope (0089 pedestrian, 0989 the server may also selectively control data flow from an autonomous vehicle based on the number of cars determined to be traveling within a group along a roadway. For example, where a group of autonomous vehicles (e.g., two or more vehicles) is determined to be traveling within a certain proximity of one another (e.g., within 100 meters, 1 km, or any other suitable proximity envelope)).
Claim(s) 44, 50, 51, 52, 53, 56 and 57
Claim(s) 44, 50, 51, 52, 53, 56 and 57 recite(s) subject matter similar to that/those of claim(s) 29, 37, 39, 29, 31, 37 and 39, respectively, and is/are rejected under the same grounds.
Claims 31-35, 45-49, 54 and 55 are rejected under 35 USC 103 as being unpatentable over Braunstein in view of Berntorp, in further view of US20190258918 (“Wang”).
Claim 31
Braunstein fails to disclose wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to: determine first rewards associated with the plurality of first potential navigational actions; and determine second rewards associated with the plurality of second potential navigational actions, wherein the one of the plurality of first potential navigational actions is selected based on the first rewards and the second rewards. However, Braunstein does disclose the plurality of navigational actions (0605). Furthermore, Wang teaches a system of controlling an autonomous vehicle (0019), including: wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to:
determine first rewards associated with the plurality of first potential navigational actions (0041 For example, each trajectory in the replay memory 150 may include the same fixed number of time steps. The trajectory data at a given time step identifies: (i) an observation characterizing a state of the environment, (ii) an action performed by the agent in response to the observation, (iii) a reward received in response to the agent performing the action, and (iv) the score distribution for the actions in the set of actions that was used in determining which action to perform in response to the observation.); and
determine second rewards associated with the plurality of second potential navigational actions, wherein the one of the plurality of first potential navigational actions is selected based on the first rewards and the second rewards (0041 For example, each trajectory in the replay memory 150 may include the same fixed number of time steps. The trajectory data at a given time step identifies: (i) an observation characterizing a state of the environment, (ii) an action performed by the agent in response to the observation, (iii) a reward received in response to the agent performing the action, and (iv) the score distribution for the actions in the set of actions that was used in determining which action to perform in response to the observation.).
Braunstein and Wang both disclose autonomous vehicle navigational action selection systems. Thus, it would have been obvious to one having ordinary skill in the art before the effective filing date of Applicant's invention and with a reasonable expectation of success to apply the known element(s) of Wang to the known system of Braunstein, the latter having been ready for improvement. The combination would have done no more than yield the predictable results of wherein the memory further includes instructions that when executed by the circuitry cause the at least one processor to: determine first rewards associated with the plurality of first potential navigational actions; and determine second rewards associated with the plurality of second potential navigational actions, wherein the one of the plurality of first potential navigational actions is selected based on the first rewards and the second rewards.
Claim 32
Braunstein fails to disclose wherein another one of the plurality of first potential navigational actions is associated with a higher one of the first rewards than the selected one of the plurality of first potential navigational actions. However, Braunstein does disclose the plurality of navigational actions (0605). Furthermore, Wang discloses a system of controlling an autonomous vehicle (0019), including:
wherein another one of the plurality of first potential navigational actions is associated with a higher one of the first rewards than the selected one of the plurality of first potential navigational actions (0040 In order for the action selection policy neural network 112 to be effectively used to select actions to be performed by the agent, a training engine 118 trains the action selection policy neural network 112 to generate policy outputs that maximize the expected cumulative reward received by the system 100, e.g., a long-term time-discounted sum of rewards received by the system 100, as a result of interactions by the agent with the environment.).
See prior art rejection of claim 31 for obviousness and reasons to combine.
Claim 33
Braunstein fails to disclose wherein determining the second rewards comprises comparing values of reward functions associated with each of the plurality of second potential navigational actions. However, Braunstein does disclose the plurality of navigational actions (0605). Furthermore, Wang discloses a system of controlling an autonomous vehicle (0019), including:
wherein determining the second rewards comprises comparing values of reward functions associated with each of the plurality of second potential navigational actions (0011 processing the observation using a critic neural network to determine a respective Q value for each of the possible actions; and determining, from the Q values and the main score distribution, a main gradient for the time step. A Q-value may provide a value for an observation-action combination, more particularly an estimate of a future reward were the action to be performed in response to the observation. The main gradient can be used to adjust the values of the policy parameters by gradient ascent or descent, for example by backpropagation through the action selection neural network.).
See prior art rejection of claim 31 for obviousness and reasons to combine.
Claim 34
Braunstein fails to disclose wherein the selection of the one of the plurality of first potential navigational actions is based on a comparison of the first rewards to the second rewards. However, Braunstein does disclose the plurality of navigational actions (0605). Furthermore, Wang discloses a system of controlling an autonomous vehicle (0019), including:
wherein the selection of the one of the plurality of first potential navigational actions is based on a comparison of the first rewards to the second rewards (0022 the described techniques can effectively train a neural network to select actions to be performed by an agent from a continuous action space. Effectively training a neural network to select actions from a continuous action space can be highly advantageous for complex tasks, e.g., tasks that include controlling a robot or an autonomous vehicle., 0040 In order for the action selection policy neural network 112 to be effectively used to select actions to be performed by the agent, a training engine 118 trains the action selection policy neural network 112 to generate policy outputs that maximize the expected cumulative reward received by the system 100, e.g., a long-term time-discounted sum of rewards received by the system 100, as a result of interactions by the agent with the environment.).
See prior art rejection of claim 31 for obviousness and reasons to combine.
Claim 35
Braunstein fails to disclose wherein the one of the plurality of first potential navigational actions is selected based on a difference between two of the first rewards. However, Braunstein does disclose the plurality of navigational actions (0605). Furthermore, Wang discloses a system of controlling an autonomous vehicle (0019), including:
wherein the one of the plurality of first potential navigational actions is selected based on a difference between two of the first rewards (0022 the described techniques can effectively train a neural network to select actions to be performed by an agent from a continuous action space. Effectively training a neural network to select actions from a continuous action space can be highly advantageous for complex tasks, e.g., tasks that include controlling a robot or an autonomous vehicle., 0040 In order for the action selection policy neural network 112 to be effectively used to select actions to be performed by the agent, a training engine 118 trains the action selection policy neural network 112 to generate policy outputs that maximize the expected cumulative reward received by the system 100, e.g., a long-term time-discounted sum of rewards received by the system 100, as a result of interactions by the agent with the environment.).
See prior art rejection of claim 31 for obviousness and reasons to combine.
Claim(s) 45, 46, 47, 48, 49, 54 and 55
Claim(s) 45, 46, 47, 48, 49, 54 and 55 recite(s) subject matter similar to that/those of claim(s) 31, 32, 33, 34, 35, 32 and 33, respectively, and is/are rejected under the same grounds.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Examiner Krishnan Ramesh, whose telephone number is (571)272-6407. The examiner can normally be reached Monday-Friday 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abby Flynn, can be reached at (571)272-9855. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KRISHNAN RAMESH/
Primary Examiner, Art Unit 3663