Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The Office Action is in response to the arguments filed 07/23/2025. Claims 1-20 are presently pending and are presented for examination.
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 07/23/2025 and 09/10/2026 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-4, 9-15, and 18-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Mueller et al. US 20200348664 A1 (“Mueller”).
Regarding Claim 1. Mueller teaches a machine comprising:
at least one central processing unit (CPU);
at least one graphics processing unit (GPU);
at least one digital signal processor (DSP);
at least one memory device (Paragraph 21 describes a CPU, a GPU, a DSP, and at least one memory device is mentioned in paragraph 22);
at least one radio frequency integrated circuit (RFIC) (Radio signal transmission is described in paragraph 23, using things such as RF transceivers and antennas, which reads on at least one radio frequency integrated circuit);
at least one network interface device (NID) (paragraph 37);
a plurality of perception sensors of a plurality of sensor modalities (Paragraph 32 describes not only different sensors but different sensor modalities in the form of at least radar and LIDAR);
at least one display unit (Paragraph 47 teaches a graphical user interface for displaying information on a display and/or receiving commands); and
at least one communication bus allowing for communication between components of the machine (FIG. 1 depicts a mobile communication terminal and a robot terminal. Alternatively, FIG. 5 depicts the communication between the mobile communication terminal and a microcontroller of the robot terminal),
wherein the machine is to perform one or more planning, navigation, or control operations based at least on one or more outputs of one or more neural networks computed based at least on the one or more neural networks processing sensor data generated using the plurality of perception sensors (Paragraph 33 describes how the robot terminal is operated using navigation instructions generated based on an artificial neural network. FIG. 7 depicts a flow-chart of the data collection and training process, according to an aspect of the disclosure, wherein the driving policy (control operations) are updated as an output of the training after receiving the dataset at 714, which is generated from sensor data [paragraph 61]).
Regarding Claim 2. Mueller teaches the machine of claim 1.
Mueller also teaches:
wherein the plurality of perception sensors include one or more of: one or more light detection and ranging (LiDAR) sensors; or one or more cameras (paragraph 32 describes sensors including LIDAR or a variety of cameras).
Regarding Claim 3. Mueller teaches the machine of claim 1.
Mueller also teaches:
wherein the one or more neural networks further process map information to generate at least one output of the one or more outputs (paragraph 61).
Regarding Claim 4. Mueller teaches the machine of claim 3.
Mueller also teaches:
wherein the one or more neural networks process the map information to identify different weights to apply to different sensor modalities of the plurality of sensor modalities as part of computing at least one output of the one or more outputs (The artificial neural network may further include one or more second layers or hidden layers, the second layers or hidden layers each including one or more nodes, wherein the various nodes are configured to perform one or more weighted calculations on the sensor data according to the artificial neural network structure [paragraph 67]).
Regarding Claim 9. Mueller teaches the machine of claim 1.
Mueller also teaches:
wherein the machine is an autonomous machine or semi- autonomous machine (Various aspects of this disclosure generally relate to autonomous or semi-autonomous operation of a robot terminal using one or more sensors and one or more processors of a mobile communication terminal [paragraph 1], further confirmed to be an independently functioning and/or autonomous robot [paragraph 35]).
Regarding Claim 10. Mueller teaches the machine of claim 1.
Mueller also teaches:
wherein the one or more outputs include one or more of:
one or more characteristics of an area around the machine;
one or more predictions about behavior of objects around the machine;
one or more plans that dictate planned movements of the machine;
a route of travel for the machine (A trained machine learning model may be used during an inference phase to make predictions or decisions based on input data. In some aspects, the trained machine learning model may be used to generate additional training data. An additional machine learning model may be adjusted during a second training phase based on the generated additional training data. A trained additional machine learning model may be used during an inference phase to make predictions or decisions based on input data [paragraph 24]. Training may include iterating through training instances and using an objective function to teach the model to predict the output for new inputs (illustratively, for inputs not included in the training set) [paragraph 26]. The one or more artificial neural networks may be configured to output an environment parameter based on the image sensor data input. The environment parameter may be any factor present in or associated with an environment of the mobile communication terminal. Furthermore, because the mobile communication terminal is to be mounted on/in or otherwise physically connected to the robot terminal, the environment parameter also represents a parameter of an environment of the robot terminal. Without limitation, the environment parameter may include one or more gestures (e.g. handwaving, providing a “stop” signal with the hands, providing one or more hand signal commands, or otherwise); one or more postures (e.g. of a device, of an obstacle, or otherwise); one or more obstacles (e.g. obstacle identification, obstacle location, obstacle movements, obstacle heading, or otherwise); one or more drivable paths (navigation decisions, navigation directions, paths without obstacles, unobstructed paths, comparison of paths to determine a most favorable path, etc.); or any combinations thereof [paragraph 70]. Combined with the predictions described in paragraphs 24-26, this reads on outputting characteristics and predictions about behavior of objects around the machine, as well as drivable paths that read on a route of travel for the machine or plans that dictate planned movements of the machine); or
one or more controls that control movement of the machine (The navigation instruction may be understood as an instruction to direct the robot terminal's movement. The mobile communication terminal may transmit the navigation instruction (e.g. a wired connection or a wireless connection) to the robot terminal (e.g. to a microcontroller of the robot terminal) [paragraph 71]).
Regarding Claim 11. Mueller teaches the machine of claim 1.
Mueller also teaches:
wherein the one or more outputs are further based at least on one or more physical characteristics of the machine (The microcontroller may, for example, handle low-level actuation and/or measurements, such as wheel odometry and battery voltage [paragraph 48], both of which are physical characteristics of the machine).
Regarding Claim 12. Mueller teaches a system comprising:
at least one central processing unit (CPU);
at least one graphics processing unit (GPU);
at least one memory device (Paragraph 21 describes a CPU, a GPU, a DSP, and at least one memory device is mentioned in paragraph 22);
a plurality of perception sensors of a plurality of sensor modalities (Paragraph 32 describes not only different sensors but different sensor modalities in the form of at least radar and LIDAR);
at least one display unit (Paragraph 47 teaches a graphical user interface for displaying information on a display and/or receiving commands); and
at least one communication bus allowing for communication between components of the system (FIG. 1 depicts a mobile communication terminal and a robot terminal. Alternatively, FIG. 5 depicts the communication between the mobile communication terminal and a microcontroller of the robot terminal), wherein the system causes a machine to perform one or more planning, navigation, or control operations based at least on one or more outputs of one or more neural networks computed based at least on the one or more neural networks processing sensor data generated using the plurality of perception sensors (Paragraph 33 describes how the robot terminal is operated using navigation instructions generated based on an artificial neural network. FIG. 7 depicts a flow-chart of the data collection and training process, according to an aspect of the disclosure, wherein the driving policy (control operations) are updated as an output of the training after receiving the dataset at 714, which is generated from sensor data [paragraph 61]).
Regarding Claim 13. Mueller teaches the system of claim 12.
Mueller also teaches:
wherein the plurality of perception sensors include one or more of: one or more light detection and ranging (LiDAR) sensors; or one or more cameras (Paragraph 32).
Regarding Claim 14. Mueller teaches the system of claim 12.
Mueller also teaches:
wherein the one or more neural networks further process map information to generate at least one output of the one or more outputs (Paragraph 61).
Regarding Claim 15. Mueller teaches the system of claim 14.
Mueller also teaches:
wherein the one or more neural networks process the map information to identify different weights to apply to different sensor modalities of the plurality of sensor modalities as part of computing at least one output of the one or more outputs (The artificial neural network may further include one or more second layers or hidden layers, the second layers or hidden layers each including one or more nodes, wherein the various nodes are configured to perform one or more weighted calculations on the sensor data according to the artificial neural network structure [paragraph 67]).
Regarding Claim 18. Mueller teaches the system of claim 12.
Mueller also teaches:
wherein the machine is an autonomous machine or semi-autonomous machine (Various aspects of this disclosure generally relate to autonomous or semi-autonomous operation of a robot terminal using one or more sensors and one or more processors of a mobile communication terminal [paragraph 1], further confirmed to be an independently functioning and/or autonomous robot [paragraph 35]).
Regarding Claim 19. Mueller teaches the system of claim 12.
Mueller also teaches:
wherein the one or more outputs include one or more of:
one or more characteristics of an area around the machine;
one or more predictions about behavior of objects around the machine;
one or more plans that dictate planned movements of the machine;
a route of travel for the machine; or
one or more controls that control movement of the machine (A trained machine learning model may be used during an inference phase to make predictions or decisions based on input data. In some aspects, the trained machine learning model may be used to generate additional training data. An additional machine learning model may be adjusted during a second training phase based on the generated additional training data. A trained additional machine learning model may be used during an inference phase to make predictions or decisions based on input data [paragraph 24]. Training may include iterating through training instances and using an objective function to teach the model to predict the output for new inputs (illustratively, for inputs not included in the training set) [paragraph 26]. The one or more artificial neural networks may be configured to output an environment parameter based on the image sensor data input. The environment parameter may be any factor present in or associated with an environment of the mobile communication terminal. Furthermore, because the mobile communication terminal is to be mounted on/in or otherwise physically connected to the robot terminal, the environment parameter also represents a parameter of an environment of the robot terminal. Without limitation, the environment parameter may include one or more gestures (e.g. handwaving, providing a “stop” signal with the hands, providing one or more hand signal commands, or otherwise); one or more postures (e.g. of a device, of an obstacle, or otherwise); one or more obstacles (e.g. obstacle identification, obstacle location, obstacle movements, obstacle heading, or otherwise); one or more drivable paths (navigation decisions, navigation directions, paths without obstacles, unobstructed paths, comparison of paths to determine a most favorable path, etc.); or any combinations thereof [paragraph 70]. Combined with the predictions described in paragraphs 24-26, this reads on outputting characteristics).
Regarding Claim 20. Mueller teaches an autonomous or semi-autonomous machine comprising:
at least one central processing unit (CPU);
at least one graphics processing unit (GPU);
at least one memory device (Paragraph 21 describes a CPU, a GPU, a DSP, and at least one memory device is mentioned in paragraph 22);
a plurality of perception sensors of a plurality of sensor modalities (Radio signal transmission is described in paragraph 23, using things such as RF transceivers and antennas, which reads on at least one radio frequency integrated circuit);
at least one display unit (Paragraph 47 teaches a graphical user interface for displaying information on a display and/or receiving commands); and
at least one communication bus allowing for communication between components of the autonomous or semi-autonomous machine (FIG. 1 depicts a mobile communication terminal and a robot terminal. Alternatively, FIG. 5 depicts the communication between the mobile communication terminal and a microcontroller of the robot terminal), wherein the autonomous or semi-autonomous machine performs one or more planning, navigation, or control operations based at least on one or more outputs of one or more neural networks computed based at least on the one or more neural networks processing sensor data generated using the plurality of perception sensors (Paragraph 33 describes how the robot terminal is operated using navigation instructions generated based on an artificial neural network. FIG. 7 depicts a flow-chart of the data collection and training process, according to an aspect of the disclosure, wherein the driving policy (control operations) are updated as an output of the training after receiving the dataset at 714, which is generated from sensor data [paragraph 61]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5-6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Mueller et al. US 20200348664 A1 (“Mueller”) as applied to claims 3 and 14 above, and further in view of Nabatchian et al. US 20210347378 A1 (“Nabatchian”).
Regarding Claim 5. Mueller teaches the machine of claim 3.
Mueller does not teach:
wherein the map information corresponds to a high-definition (HD) map of a geographical region.
However, Nabatchian teaches:
wherein the map information corresponds to a high-definition (HD) map of a geographical region (Paragraph 69 describes autonomous vehicle using high-definition (HD) maps to augment point cloud data and using such a data-rich map as input to one or more neural networks [paragraph 69]).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Mueller with wherein the map information corresponds to a high-definition (HD) map of a geographical region as taught by Nabatchian so as to allow the map information to include greater detail to train the neural network.
Regarding Claim 6. Mueller in combination with Nabatchian teaches the machine of claim 5.
Mueller also teaches:
wherein the map includes one or more of:
a landmark map that provides one or more of a geometric or semantic description of elements included in the geographical region (the environment of the mobile communication terminal device includes one or more landmarks, maps, static objects, moving objects, living beings, or any combination thereof [paragraph 87]).
Mueller does not teach:
wherein the map is an HD map that includes:
an occupancy map that provides one or more spatial representations of one or more roads included in the geographical region and physical objects around the one or more roads.
However, Nabatchian teaches:
wherein the map is an HD map that includes:
an occupancy map that provides one or more spatial representations of one or more roads included in the geographical region and physical objects around the one or more roads (Paragraphs 69-70).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Mueller with wherein the map is an HD map that includes: an occupancy map that provides one or more spatial representations of one or more roads included in the geographical region and physical objects around the one or more roads as taught by Nabatchian so as to allow the system to utilize occupancy data to further train the neural network, identifying roads with higher levels of traffic and incorporating them into the navigation of the vehicle.
Regarding Claim 16. Mueller teaches the system of claim 14.
Mueller does not teach:
wherein the map information corresponds to a high-definition (HD) map of a geographical region.
However, Nabatchian teaches:
wherein the map information corresponds to a high-definition (HD) map of a geographical region (Paragraphs 69-70).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Mueller with wherein the map information corresponds to a high-definition (HD) map of a geographical region as taught by Nabatchian so as to allow the map information to include greater detail to train the neural network.
Claim(s) 7-8 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Mueller et al. US 20200348664 A1 (“Mueller”) as applied to claims 3 and 14 above, and further in view of Schrier et al. US 20170206426 (“Schrier”).
Regarding Claim 7. Mueller teaches the machine of claim 3.
Mueller does not teach:
wherein the map information corresponds to a low-resolution map of a geographical region.
However, Schrier teaches:
wherein the map information corresponds to a low-resolution map of a geographical region (In one embodiment, a saliency map may have a lower resolution than the image at 200 of FIG. 2 [paragraph 34]).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Mueller with wherein the map information corresponds to a low-resolution map of a geographical region as taught by Schrier because low- resolution saliency maps can still be very effective and can also reduce processing workload or processing delay as taught by Schrier in paragraph 34.
Regarding Claim 8. Mueller in combination with Schrier teaches the machine of claim 7.
Mueller does not teach:
wherein the low-resolution map describes one or more of:
structures within the geographical region, or geological features of the geographical region.
However, Schrier teaches:
wherein the low-resolution map describes one or more of:
structures within the geographical region, or geological features of the geographical region (FIG. 2 shows a number of structures within the geographical region of the low-resolution map).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Mueller with wherein the low-resolution map describes one or more of: structures within the geographical region, or geological features of the geographical region as taught by Schrier so as to allow the system to incorporate data regarding structures or features in the geographical region into the training data.
Regarding Claim 17. Mueller teaches the system of claim 14.
Mueller does not teach:
wherein the map information corresponds to a low- resolution map of a geographical region.
However, Schrier teaches:
wherein the map information corresponds to a low-resolution map of a geographical region (In one embodiment, a saliency map may have a lower resolution than the image at 200 of FIG. 2 [paragraph 34]).
It would have been obvious to one of ordinary skill in the art at the time the invention was filed to modify the invention of Mueller with wherein the map information corresponds to a low-resolution map of a geographical region as taught by Schrier because low- resolution saliency maps can still be very effective and can also reduce processing workload or processing delay as taught by Schrier in paragraph 34.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AARON G CAIN whose telephone number is (571)272-7009. The examiner can normally be reached Monday: 7:30am - 4:30pm EST to Friday 7:30pm - 4:30am.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wade Miles can be reached at (571) 270-7777. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AARON G CAIN/Examiner, Art Unit 3656
Prosecution Insights