Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR1.114. Applicant's submission filed on 12/23/25 has been entered.
Response to Amendment and Arguments
The amendment filed 12/23/2025 has been entered. Claims 1-3, 10-22 remain pending in the application. Amendments have overcome the previous rejection under 35 USC 112.
Applicant’s arguments with respect to the rejection(s) under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground(s) of rejection is made in view of US20200183011A1 Lin et al ("Lin") and US11521396B1 Jain et al ("Jain").
Claim Objections
Claim 22 is objected to because of the following informalities: there appears to be a typographical mistake. The claim recites "method claim 1" whereas the Examiner believes the Applicant to have intended the claim to recite "method of claim 19". Appropriate correction is required.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1, 11, 16, 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over US20230241772A1 Schillinger et al ("Schillinger") in view of US20200183011A1 Lin et al ("Lin").
As per claims 1, 19, and 20, Schillinger teaches the limitations of the apparatus, method, and non-transitory computer readable medium:
An apparatus for performing a task, the task being a sequence of actions performed to achieve a goal, the apparatus comprising: at least one sensor for obtaining observations of the apparatus; a controller configured to receive a control signal to move said apparatus; and a processor (Schillinger at least the abstract, [0046-0048], [0025])
processor being configured to: receive information concerning the goal; determine the sequence of actions to reach said goal, the sequence of actions being subject to at least one constraint; (Schillinger at least the abstract: “adherence to at least one condition”)
provide a control signal to said controller for the next action in said sequence of actions, wherein said processor is configured to determine the sequence of actions by processing observations received by said sensors to obtain information concerning the at least one constraint (Schillinger at least [0013]: “steps”, abstract)
performing stochastic optimisation to determine the sequence of actions, the stochastic optimisation receiving an initial estimate of the next action. (Schillinger at least [0057], [0044-0046])
the task comprises moving a vehicle to achieve the goal, the sequence of actions being a sequence of movements (Schillinger at least [0013], FIG. 1)
Schillinger does not explicitly disclose the constraint being represented as a cost in the stochastic optimization, however Schillinger does teach the equivalent of the cost function to model the relationship between the parameters of the model and the predicted (expected) target values which is part of the randomly selected variables (stochastic) modelling (Schillinger at least [0057]).
It would be obvious, to one of ordinary skill in the art before the filing date of the invention, to have implemented the teachings of Schillinger as being capable of performing the task as claimed.One of ordinary skill in the art would be motivated to apply the teachings of Schillinger to improve a robot control model for a particular skill (Schillinger [0053]).
Schillinger does not disclose:
the processor is configured to produce a plurality of occupancy maps, including a dynamic occupancy map which represents a first probability of a position being occupied by a dynamic obstacle at a time instance t and a static occupancy map which represents a second probability of a position being occupied by a static obstacle at the time instance t, and provide the control signal based on a probability distribution of a likelihood of a predicted position of the apparatus being occupied at the time t, the probability distribution being provided by the first probability and the second probability.
Lin teaches the aforementioned limitations (Lin at least [0063-0093]: “correspond to the static object category. Assuming that the probabilities that a pixel belongs to Vehicle, Motorcycle, Bicycle, Human, Ground, Tree, Streetlight, Traffic Light, Curb, and Fence are 94%, 2%, 1%, 1%, 0.8%, 0.2%, 0.1%, 0.2%, 0.6%, and 0.1%, respectively, the probabilities that the grid corresponding to the pixel belongs to the dynamic object category, the static object category, and the ground category are 98%, 1.2%, and 0.8%, respectively.”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Lin with a reasonable expectation of success. One of ordinary skill would have been motivated to combine these references in order to enhance the stability of the state of an occupancy grid map (Lin [0027]).
As per claim 11, Schillinger teaches the invention as described above. Schillinger additionally teaches:
there are a plurality of constraints, the processor being configured to apply a weighting to each costs derived from the plurality of constraints such that costs are applied with variable weightings. (Schillinger at least [0056])
As per claim 16, Schillinger teaches the invention as described above. Schillinger additionally teaches:
the processor is configured to input an estimated sequence of actions to reach the goal as an input estimate into a stochastic optimiser. (Schillinger at least [0057]: “parameter values can be randomly selected”)
Claim(s) 2, 10, 12-15, 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schillinger and Lin in view of US20220075383A1 Morad et al ("Morad").
Regarding claim 2, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
the initial estimate is obtained from a reinforcement learning policy.
However, Morad teaches the aforementioned limitation (Morad at least the abstract, [0032], [0057], [0179]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Morad with a reasonable expectation of success. One of ordinary skill would have been motivated to combine these references in order to more efficiently train a model and reduce a number of actions (Morad [0033]).
Regarding claim 10, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
at least one of the constraints requires the apparatus to move the shortest distance to reach the goal.
However, Morad teaches the aforementioned limitation (Morad at least [0166]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Morad with a reasonable expectation of success. The motivation to combine these references is the same as above in claim 2.
Regarding claim 12, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
the input to the reinforcement learning module is a hidden state of a recurrent neural network "RNN", the RNN being used to produce predictions of at least one future observation.
However, Morad teaches the aforementioned limitation (Morad at least the abstract, [0171]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Morad with a reasonable expectation of success. The motivation to combine these references is the same as above in claim 2.
Regarding claim 13, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
an input to the RNN is a latent representation of the observations.
However, Morad teaches the aforementioned limitation (Morad at least [0066-0067], [0119], [0237], [0247]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Morad with a reasonable expectation of success. The motivation to combine these references is the same as above in claim 2.
Regarding claim 14, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
a further input to the RNN is the goal.
However, Morad teaches the aforementioned limitation (Morad at least [0277], [0288]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Morad with a reasonable expectation of success. The motivation to combine these references is the same as above in claim 2.
Regarding claim 15, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
the processor is configured to provide probabilities relating to predictions of future observations from the RNN to a stochastic optimiser.
However, Morad teaches the aforementioned limitation (Morad at least [0042], [0160], [0164], [0175], [0182], [0262]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Morad with a reasonable expectation of success. The motivation to combine these references is the same as above in claim 2.
Regarding claim 17, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
the observations are LiDAR data.
However, Morad teaches the aforementioned limitation (Morad at least [0157]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Morad with a reasonable expectation of success. The motivation to combine these references is the same as above in claim 2.
Regarding claim 18, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
the reinforcement learning policy is a proximal policy optimisation algorithm.
However, Morad teaches the aforementioned limitation (Morad at least [0186]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Morad with a reasonable expectation of success. The motivation to combine these references is the same as above in claim 2.
Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schillinger and Lin in view of US11195418B1 Hong et al ("Hong").
Regarding claim 3, Schillinger in combination with the other reference teaches the invention as described above. Schillinger does not disclose:
the processor is configured to process observations concerning the at least one constraint using a plurality of parallel processing branches, such that each branch is allocated to a different constraint.
However, Hong teaches the aforementioned limitation (Hong at least col 8 lines 20-25).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Hong with a reasonable expectation of success. One of ordinary skill would have been motivated to combine these references in order to improve functioning of a computer device (Hong col 5).
Claim(s) 21, 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Schillinger and Lin in view of US11521396B1 Jain et al ("Jain", previously cited).
As per claims 21 and 22, Schillinger in combination with the other reference teaches the inventions as described above. Schillinger [0068] provides for implementing the teachings on a vehicle, however, Schillinger does not disclose:
the task comprises moving a vehicle to achieve the goal, the sequence of actions being a sequence of movements, the task is a navigation task and the goal is a location, and the at least one constraint comprises navigating to avoid dynamic obstacles which are moving obstacles and navigating to avoid static obstacles
Jain teaches the aforementioned limitations (Jain at least col 9: “A fully autonomous (e.g., self-driving) operational mode can be one in which the vehicle 102 can provide driving and navigational operation 30 with minimal and/or no interaction from a human driver present in the vehicle. A semi-autonomous operational mode can be one in which the vehicle 102 can operate with some interaction from a human driver present in the vehicle. Park and/or sleep modes”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine Schillinger with the aforementioned limitations taught by Jain with a reasonable expectation of success. One of ordinary skill would have been motivated to combine these references in order to improve the safety of autonomous vehicles (Jain col 3).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to OLIVER TAN whose telephone number is (703)756-4728. The examiner can normally be reached M-F 10-7.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Navid Mehdizadeh can be reached at (571) 272-7691. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/O.T./Examiner, Art Unit 3669
/NAVID Z. MEHDIZADEH/Supervisory Patent Examiner, Art Unit 3669