DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Preliminary Remarks
This is a reply to the application filed on 12/13/2024, in which, claims 1-24 remain pending in the present application with claims 1, 23, and 24 being independent claims.
When making claim amendments, the applicant is encouraged to consider the references in their entireties, including those portions that have not been cited by the examiner and their equivalents as they may most broadly and appropriately apply to any particular anticipated claim amendments.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on October 10, 2025 is in compliance with the provisions of 37 CFR 1.97 and is being considered by the Examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-24 are rejected under 35 U.S.C. 103 as being unpatentable over Versace et al. (US 20200151446 A1, hereinafter referred to as “Versace”) in view of Guzman et al. (US 20200322528 A1, hereinafter referred to as “Guzman”).
Regarding claim 1, Versace discloses a method for training a camera policy neural network that is used to control a position of a camera sensor in an environment being interacted with by a robot (see Versace, paragraph [0024]: “The processor also determines the next fixation point of the sensor system based on the location and/or identity of the object(s). In some cases, it transmits movement vector representing the saccade between the current fixation point and the next fixation point to an actuator that then actuates the sensor appropriately. For instance, the processor may cause a pan-tilt actuator to move a camera mounted on the robot so as to acquire imagery of an object from different angles and/or positions. The robot itself may move to change the sensor's field of view”), the method comprising:
obtaining data specifying one or more target sensors of the robot (see Versace, paragraph [0064]: “The environment module 120 provides both image data from the image sensor 100 and actuation data (sensor position data) from the actuator(s) 110 to the Where system 130, which in turn provides processed image data to the What system 150. The environment module 120 also provides actuation data (sensor position data) from the actuator(s) 110 to the Teacher 160, which forms part of the What pathway 170 with the What system 150”);
obtaining a first observation comprising one or more images of the environment captured by the camera sensor while at a current position (see Versace, paragraph [0064]: “FIG. 1 shows the Environment Module (120) and the Where System (130), which collectively constitute the Where Pathway (140). The environment module 120 includes an RGB image sensor 100, which may acquire still and/or video images”);
processing a camera policy input comprising
(i) the data specifying one or more target sensors of the robot (see Versace, paragraph [0003]: “A mobile robot may fuse lidar, radar, and/or other data with visible image data in order to more accurately identify and locate obstacles in its environment”);
adjusting the current position of the camera sensor based on the camera control action (see Versace, paragraph [0024]: “the processor may cause a pan-tilt actuator to move a camera mounted on the robot so as to acquire imagery of an object from different angles and/or positions. The robot itself may move to change the sensor's field of view”); and
obtaining a second observation comprising one or more images of the environment captured by the camera sensor while at the adjusted position (see Versace, paragraph [0064]: “The environment module 120 includes an RGB image sensor 100, which may acquire still and/or video images, whose field of view can be shifted, narrowed, and/or expanded with one or more actuators 110, including but not limited to zoom lenses, tip/tilt stages, translation stages, etc. The environment module 120 provides both image data from the image sensor 100 and actuation data (sensor position data) from the actuator(s) 110 to the Where system 130, which in turn provides processed image data to the What system 150. The environment module 120 also provides actuation data (sensor position data) from the actuator(s) 110 to the Teacher 160, which forms part of the What pathway 170 with the What system 150”).
Regarding claim 1, Versace discloses all the claimed limitations with the exception of (ii) the first observation that comprises one or more images captured by the camera sensor using the camera policy neural network to generate a camera policy output that defines a camera control action for adjusting the position of the camera sensor; generating, from the second observation, a respective prediction for each of the one or more target sensors that characterizes sensor readings generated by the target sensor; generating, for each target sensor, a respective reward for the camera policy neural network from an error in the respective prediction for the target sensor; and training the camera policy neural network using the rewards for the one or more target sensors.
Guzman from the same or similar fields of endeavor discloses (ii) the first observation that comprises one or more images captured by the camera sensor using the camera policy neural network to generate a camera policy output that defines a camera control action for adjusting the position of the camera sensor (see Guzman, paragraph [0023]: “The RL stage trains internal one or more policy neural networks (e.g., a third neural network, not shown), the neural network in the space representation stage 50 a and the neural network in the trajectory prediction stage 50 b to maximize the likelihood of collecting the largest number of rewards”);
generating, from the second observation, a respective prediction for each of the one or more target sensors that characterizes sensor readings generated by the target sensor (see Guzman, paragraph [0030]: “tracking information is updated at block 82 and block 84 calculates the predicted direction and speed (e.g., trajectory) of the individual. Block 86 may forecast the best camera at a future moment in time (e.g., time t). In one example, block 88 schedules the best camera to capture the individual at the future moment in time”);
generating, for each target sensor, a respective reward for the camera policy neural network from an error in the respective prediction for the target sensor (see Guzman, paragraph [0023]: “a reinforcement learning (RL) stage 50 c (e.g., control system) may operate the cameras through the camera operation system and inform the other stages about the usefulness of the outputs (e.g., predicted trajectories) from those stages based on rewards”); and
training the camera policy neural network using the rewards for the one or more target sensors (see Guzman, paragraph [0023]: “provide feedback to the other three neural networks (e.g., indicating whether the system was able to detect the face, that is the reward of the reinforcement learning). In such a case, the other three neural networks may adjust accordingly to achieve better rewards in the future”).
Therefore it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize the teachings as in Guzman with the teachings as in Versace. The motivation for doing so would ensure the system to have the ability to use the system and method disclosed in Guzman to operate the cameras through the camera operation system and inform the other stages about the usefulness of the outputs from those stages based on rewards; to train internal one or more policy neural networks wherein the neural network in the space representation stage and the trajectory prediction stage to maximize the likelihood of collecting the largest number of rewards; to adjust neural networks accordingly to achieve better rewards in the future; and to calculate the predicted direction and speed to predict the best camera at a future moment in time to capture thus comprising one or more images captured by the camera sensor using the camera policy neural network to generate a camera policy output that defines a camera control action for adjusting the position of the camera sensor; generating a respective prediction for each of the one or more target sensors that characterizes sensor readings generated by the target sensor; generating a respective reward for the camera policy neural network from an error in the respective prediction for the target sensor and training the camera policy neural network using the rewards for the one or more target sensors in order to train neural network to learn active vision skills so that the neural network policy can be used to control a position of a camera sensor in an environment being interacted with by a robot.
Regarding claim 2, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 1, wherein the camera sensor is part of the robot (see Versace, paragraph [0052]: “a robot with a camera”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 3, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 1, wherein the camera sensor is external to the robot within the environment (see Versace, paragraph [0056]: “The Environment Module (100) abstracts interactions between the vision system and the environment, which can be a virtual environment or a real environment sampled by a fix/pan-tilt camera, a robot-mounted camera, or other visual (or non-visual) sensory system. This module delivers a visual image to the visual system and executes camera movement commands, which emulate human eye movements. The environment module allows OpenEye to interact with the environment: virtual or real, static or dynamic, real time or prerecorded”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 4, the combination teachings of Versace and Guzman as discussed above also disclose the method of any preceding claimclaim1, wherein the camera sensor is a foveal camera (see Versace, paragraph [0058]: “OpenEye's Where System generates camera movements in order sample an image by foveation on the spatial location it selects as the most salient, where saliency can be determined by sensory input or semantic (What System) information”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 5, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 4, wherein the foveal camera comprises a plurality of cameras (see Versace, paragraph [0052]: “identifying, learning, localizing, and tracking objects in visual scenes provided by cameras”) with different fields of view robot (see Versace, paragraph [0024]: “the processor may cause a pan-tilt actuator to move a camera mounted on the robot so as to acquire imagery of an object from different angles and/or positions”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 6, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim1, wherein the respective prediction is a prediction of a value of a sensor reading of the target sensor at a time step at which the second observation is generated (see Guzman, paragraph [0022]: “a trajectory prediction stage 50 b may use a recurrent neural network (e.g., second neural network, not shown) to predict the subsequent movement of the individual in a trajectory tiling 54 (54 a-54 c) that includes predicted place cell activity 54 c corresponding to a future trajectory 55, in addition to the observed place cell activity 54 a and the inactive place cells 54 b”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 7, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim1, wherein the respective prediction is a prediction of a return generated from at least values of sensor readings of the target sensor at each of one or more time steps after the time step at which the second observation is generated (see Versace, paragraph [0172]: “Foveation memory inhibits foveations at the locations that have been foveated in the past. After making a camera movement, OpenEye sets foveation activity at the maximum value (255), this activity decays with each foveation and eventually, when it decays to 0, the location is enabled for new foveations”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 8, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim1, wherein generating, from the second observation, a respective prediction for each of the one or more target sensors that characterizes sensor readings generated by the target sensor comprises:
processing a predictor input comprising the second observation using a sensor prediction neural network to generate a predictor output comprising the respective predictions for each of the one or more target sensors (see Guzman, paragraph [0027]: “Illustrated processing block 62 provides for detecting an unidentified individual at a first location along a trajectory in a scene based on a video feed of the scene, wherein the video feed is associated with a stationary (e.g., fixed) camera. In an embodiment, block 62 includes predicting the trajectory based on the video feed”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 9, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 8, further comprising: training the sensor prediction neural network using the errors in the respective predictions for the one or more target sensors (see Guzman, paragraph [0023]: “The RL stage trains internal one or more policy neural networks (e.g., a third neural network, not shown), the neural network in the space representation stage 50 a and the neural network in the trajectory prediction stage 50 b to maximize the likelihood of collecting the largest number of rewards”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 10, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 9, wherein:
the robot comprises a plurality of sensors that include the one or more target sensors (see Versace, paragraph [0003]: “A mobile robot may also use other sensors, such as radar or lidar, to acquire additional data about its environment… A mobile robot may fuse lidar, radar, and/or other data with visible image data in order to more accurately identify and locate obstacles in its environment”),
the predictor output comprises a respective prediction for each of the plurality of sensors (see Guzman, paragraph [0064]: “train a second neural network to predict trajectories of the unidentified individuals based on the simulation data, and train a third neural network to select non-stationary cameras and automatically instruct the selected non-stationary cameras to adjust at least one of the one or more settings based on the simulation data”), and
training the sensor prediction neural network comprises training the sensor prediction neural network using errors in the respective predictions for each of the plurality of sensors (see Guzman, paragraph [0023]: “The RL stage trains internal one or more policy neural networks (e.g., a third neural network, not shown), the neural network in the space representation stage 50 a and the neural network in the trajectory prediction stage 50 b to maximize the likelihood of collecting the largest number of rewards”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 11, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim1, wherein the target sensors comprise one or more proprioceptive sensors of the robot (see Versace, paragraph [0071]: “the processors may communicate with sensors, actuators, and other devices and components on or in the robot via a suitable communications link, such as a radio-frequency or optical communications link”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 12, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim1, wherein the action specifies a target velocity for each of one or more actuators of the camera sensor (see Guzman, paragraph [0030]: “tracking information is updated at block 82 and block 84 calculates the predicted direction and speed (e.g., trajectory) of the individual. Block 86 may forecast the best camera at a future moment in time (e.g., time t). In one example, block 88 schedules the best camera to capture the individual at the future moment in time”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 13, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim1, wherein training the camera policy neural network using the rewards for the one or more target sensors comprises training the camera policy neural network through reinforcement learning (see Guzman, paragraph [0030]: “trains a third neural network (e.g., policy neural network in a reinforcement learning stage) to select non-stationary cameras based on the predicted trajectories and automatically instruct the selected non-stationary cameras to adjust at least one of the one or more settings based on the simulation data”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 14, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim1, wherein training the camera policy neural network through reinforcement learning comprises training the camera policy neural network jointly with a camera critic neural network (see Guzman, paragraph [0038]: “Block 108 may re-train the first neural network, the second neural network, and the third neural network based on real-time reinforcement data ... the trajectory may be predicted via the second neural network. In an embodiment, the non-stationary camera is selected via the third neural network and the selected non-stationary camera is automatically instructed via the third neural network. The illustrated method 100 therefore further enhances performance by enabling accurate identifications to be made upon deployment of the system”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 15, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim1, wherein the robot further comprises one or more controllable elements (see Versace, paragraph [0086]: “Sensor Movement Actuator implements sensor (e.g., camera) movement commands if they are supported by the environment”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 16, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 15, wherein each of the controllable elements are controlled using a respective fixed policy during the training of the camera policy neural network (see Guzman, paragraph [0035]: “the simulator is used to generate millions of training examples of people moving across the area of interest. By using domain randomization techniques on the camera models, camera layouts and adding noise to the generated trajectories, the simulator generates samples that make the RL controller more robust to measurement errors or changes in the camera projection matrices due to aging of the lenses. The usage of the simulator with domain randomization provides a pre-trained system ready to deploy in the real environment”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 17, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 15, wherein, during the training of the camera policy neural network, each of the controllable elements are controllable using a robot policy neural network that receives inputs comprising one or more images generated by the camera sensor (see Versace, paragraph [0058]: “OpenEye's Where System generates camera movements in order sample an image by foveation on the spatial location it selects as the most salient, where saliency can be determined by sensory input or semantic (What System) information”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 18, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 17, wherein the robot policy neural network is trained on external rewards for a specified task during the training of the camera policy neural network (see Guzman, paragraph [0023]: “a reinforcement learning (RL) stage 50 c (e.g., control system) may operate the cameras through the camera operation system and inform the other stages about the usefulness of the outputs (e.g., predicted trajectories) from those stages based on rewards”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 19, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 18, wherein the training of the camera policy neural network is performed as an auxiliary task during the training of the robot policy neural network (see Guzman, paragraph [0022]: “Given the partial observed trajectory 53 through the space, a trajectory prediction stage 50 b may use a recurrent neural network (e.g., second neural network, not shown) to predict the subsequent movement of the individual in a trajectory tiling 54 (54 a-54 c) that includes predicted place cell activity 54 c corresponding to a future trajectory 55, in addition to the observed place cell activity 54 a and the inactive place cells 54 b”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 20, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 15, further comprising:
after the training of the camera policy neural network (see Guzman, paragraph [0035]: “The usage of the simulator with domain randomization provides a pre-trained system ready to deploy in the real environment. After deployment, the system may re-train the neural networks with real examples (e.g., real-time reinforcement data”):
training, using the trained camera policy neural network, a robot policy neural network that receives inputs comprising one or more images generated by the camera sensor to control each of the one or more controllable elements using external rewards for one or more specified tasks (see Guzman, paragraph [0023]: “The RL stage trains internal one or more policy neural networks (e.g., a third neural network, not shown), the neural network in the space representation stage 50 a and the neural network in the trajectory prediction stage 50 b to maximize the likelihood of collecting the largest number of rewards. For example, a reward scheme 58 might provide the largest number of identifications for each person traversing the space”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 21, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 20, wherein training, using the trained camera policy neural network, a robot policy neural network that receives inputs comprising one or more images generated by the camera sensor to control each of the one or more controllable elements using external rewards for one or more specified tasks comprises:
using the trained camera policy neural network to generate training data for the training of the robot policy neural network (see Versace, paragraph [0053]: “The novel method presented herein can have application in designing software to either extract information or control mobile robots or cameras. In particular, the method allows these machines to increase their knowledge base over time without the need to retrain the system on the entire knowledge base”).
The motivation for combining the references has been discussed in claim 1 above.
Regarding claim 22, the combination teachings of Versace and Guzman as discussed above also disclose the method of claim 15, wherein the one or more controllable elements comprise one or more manipulators (see Versace, paragraph [0024]: “cause a pan-tilt actuator to move a camera mounted on the robot so as to acquire imagery of an object from different angles and/or positions”).
The motivation for combining the references has been discussed in claim 1 above.
Claim 23 is rejected for the same reasons as discussed in claim 1 above. In addition, the combination teachings of Versace and Guzman as discussed above also disclose a system comprising: one or more computers (see Versace, paragraph [0071]: “the processors (include GPUs) may be located in one or more smart phones, tablets, and/or single board computers (SBCs)”); and
one or more storage devices communicatively coupled to the one or more computers, wherein the one or more storage devices store instructions that, when executed by the one or more computers, cause the one or more computers to perform operations for training a camera policy neural network that is used to control a position of a camera sensor in an environment being interacted with by a robot (see Versace, paragraph [0070]: “The robot may include one or more processors that are coupled to the memory and configured to execute the instructions so as to implement the What and Where systems, including the individual modules shown in FIGS. 1-4. For example, the robot may execute the instructions with a central processing unit (CPU) and a graphics processing unit (GPU)”).
Claim 24 is rejected for the same reasons as discussed in claim 1 above. In addition, the combination teachings of Versace and Guzman as discussed above also disclose one or more non-transitory computer storage media storing instructions that when executed by one or more computers cause the one or more computers to perform operations for training a camera policy neural network that is used to control a position of a camera sensor in an environment being interacted with by a robot (see Versace, paragraph [0230]: “various inventive concepts may be embodied as a computer readable storage medium (or multiple computer readable storage media) (e.g., a computer memory, one or more floppy discs, compact discs, optical discs, magnetic tapes, flash memories, circuit configurations in Field Programmable Gate Arrays or other semiconductor devices, or other non-transitory medium or tangible computer storage medium) encoded with one or more programs that, when executed on one or more computers or other processors, perform methods that implement the various embodiments of the invention discussed above”).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NIENRU YANG whose telephone number is (571)272-4212. The examiner can normally be reached Monday-Friday 10AM-6PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, THAI TRAN can be reached at 571-272-7382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
NIENRU YANG
Examiner
Art Unit 2484
/NIENRU YANG/Examiner, Art Unit 2484
/THAI Q TRAN/Supervisory Patent Examiner, Art Unit 2484