DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to applicant’s amendment/remarks filed 01/26/2026. Claims 1, 12-13, and 19-20 have been amended. No claims have been cancelled and no claims have been newly added. Accordingly, claims 1-20 are pending.
Response to Arguments
Applicant’s arguments, see page 7 filed 03/28/2025, with respect to the 35 U.S.C. 101 rejection with respect to independent claim 1 have been fully considered and are persuasive. The 35 U.S.C. 101 rejection of claim 1 has been withdrawn, since the vehicle is operated autonomously or semi-autonomously within an environment provided by the two-dimensional map that is created from this method.
Applicant’s arguments with respect to claims 1-20 under 35 U.S.C. 103 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. A new ground of rejection is made in view of Lukierski et al. US20180189565A1 and Kamarulzaman et al. “Performance Analysis of the Microsoft Kinect Sensor for 2D Simultaneous Localization and Mapping (SLAM) Techniques, 2014” (henceforth Kamarulzaman). See the new 35 U.S.C. 103 rejection below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Lukierski et al. US20180189565A1 (henceforth Lukierski) further in view of Kamarulzaman et al. “Performance Analysis of the Microsoft Kinect Sensor for 2D Simultaneous Localization and Mapping (SLAM) Techniques, 2014” (henceforth Kamarulzaman)
Regarding claim 1,
Lukierski discloses:
A method of creating a map of an environment surrounding a vehicle (See Title and Abstract) comprising the steps of:
obtaining vehicle odometry information from at least one vehicle sensor; (See at least Para. 0075, “odometry data from the robotic device may be used to constrain an optimization function. Odometry is the use of data from motion sensors to estimate a change in position over time. Odometry data may arise from the at least one movement actuator of the robotic device, e.g. tracking the position of wheels 115 or tracks 165 in FIGS. 1A and 1B.” Vehicle odometry information is obtained from at least one vehicle sensor.)
obtaining images including objects within an environment from at least one camera mounted on the vehicle; (See at least Para. 0067, “At block 610, image data is obtained from a monocular multi-directional camera device coupled to the robotic device.” Images that includes objects (Para. 0069, “objects in the space”) within an environment is obtained from a camera mounted on the vehicle.)
creating a three-dimensional depth map of the environment based on images obtained from the camera and the vehicle odometry information;
(See at least Para. 0069, “ At block 630, a set of depth values are estimated by evaluating a volumetric function of the image data from block 610 and the pose data from block 620. Each depth value in this case represents a distance from the monocular multi-directional camera device to an object in the space”. Further see Para. 0075, wherein odometry data can be used for determining the pose data. Therefore, since the depth map is created based on image data and pose data (wherein pose data contains vehicle odometry data), then the 3D depth map is created based on images obtained from the camera and vehicle odometry information.)
and generating at least one navigational instruction using the two-dimensional map to operate the vehicle autonomously or semi-autonomously within an environment provided by the two-dimensional map. (See at least Para. 0042, “the navigation engine may be configured to use an object occupancy map to determine a cleaning pattern for unoccupied areas of the space and instruct activation of the cleaning element 180 according to the cleaning pattern. For example, a vacuum device may be activated to clean an area of free-space within a room, as indicated by the object occupancy map, wherein the cleaning robotic device navigates obstacles within the room using the object occupancy map.” The occupancy map (i.e. a 2D map created from the depth map) is used to generate at least one navigational instruction to operate the vehicle autonomously or semi-autonomously within an environment provided by the 2D map.)
wherein the two-dimensional map shows open space and obstacles at least some of which have been carried over from the images, to the three-dimensional depth map and hence to the two-dimensional map.(See at least Fig. 3, Fig. 6, and Para. 0070, “the depth values are processed to populate an object occupancy map for the space. The object occupancy map is useable by the robotic device to navigate the space. For example, in one case the object occupancy map may be used to determine a navigation path for the robotic device through unoccupied areas of the space, e.g. to avoid areas of an occupancy grid that are marked as occupied.” The occupancy map (which is created from processing the depth values in Block 640 of Fig. 6) shows open space and obstacles, which have been carried over from the images (Block 610, Fig. 6), to the 3D depth map (Block 630, Fig. 6), and hence to the 2D occupancy map (Block 640, Fig. 6).)
Lukierski does not specifically state converting the three-dimensional depth map into a two-dimensional laser scan; converting the two-dimensional laser scan into a two-dimensional map based on the laser scan of the three-dimensional depth map and information from a vehicle navigation system that is indicative of vehicle operation; wherein the two-dimensional laser scan is a simplification of the three-dimensional depth map, and wherein the two-dimensional map shows open space and obstacles at least some of which have been carried over from the images, to the three-dimensional depth map, then to the two-dimensional laser scan and hence to the two-dimensional map. However, Kamarulzaman teaches:
converting the three-dimensional depth map into a two-dimensional laser scan; (See at least Section 3.2, “The Kinect’s 3D depth data is converted into 2D laser scan-like data based on the method proposed in our previous work”. The 3D depth data is converted into a 2D laser scan.)
converting the two-dimensional laser scan into a two-dimensional map based on the laser scan of the three-dimensional depth map and information from a vehicle navigation system that is indicative of vehicle operation;
(See at least Section 3.3.1 “Gmapping” and Equation 10, wherein Gmapping requires both odometry and scan observations. Further see Section 3.3.1. “The main idea of the RBPF is to estimate the trajectory of the robot, x_1:t and the map m, given the observations, z_1:t and the odometry data, u_1:t “ and “The factorization simplifies the computations such that it allows the process to be carried out in two steps. First, the trajectory of the robot can be estimated using the odometry data and the observations. Then, the map p(m|x_1:t, z_1:t-1)can be computed since x_1:t and z_1:t are known.” The 2D laser scan is converted into a 2D map (i.e. map p(m|x_1:t, z_1:t-1)) based on the scan of the 3D depth map and information from a vehicle navigation system that is indicative of vehicle operation.)
wherein the two-dimensional laser scan is a simplification of the three-dimensional depth map: (See at least Section 3.2, equation (9), wherein equation (9) reduces a full 640x480 3D depth array to a 640x1 array by taking the minimum Z-value in each column, which is a mathematical simplification of the 3D depth map.)
wherein the two-dimensional map shows open space and obstacles at least some of which have been carried over from the images, to the three-dimensional depth map, then to the two-dimensional laser scan and hence to the two-dimensional map. (See at least Section 6, Fig. 7, wherein the Gmapping algorithm (from section 3.3.1.) produces the two-dimensional map that shows white and black pixels representing unoccupied (i.e. open space) and occupied (i.e. obstacles) areas. The two-dimensional map in Fig. 7 shows the open space and obstacles that have been carried over from the images of Kinect’s depth camera (i.e. the 3D depth map) to the two-dimensional laser scan, and hence to the two-dimensional map.)
It would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to have modified Lukierski to incorporate the teachings of Kamarulzaman to include the limitations as recited above since “the Kinect has advantages in terms of the extra dimensionality it provides (i.e., three dimensions) and its significantly lower cost” and “This aspect is very important in order to perform SLAM in a real environment where there exist objects of variable shape and size. The robot will also able to avoid certain obstacles that are typically unseen by 2D sensors” (See Section 2. “Kinect vs Laser Scanners”, Kamarulzaman). This would create a more robust system for creating a map of an environment surrounding a vehicle, by adding a step of converting the 3D depth map into a laser scan before creating the 2D map. Additionally, a person having ordinary skill in the art would have a reasonable expectation of success in combining the teachings of Lukierski and Kamarulzaman. The claimed invention is merely a combination of known elements and in combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that the results of the combination would have been predictable.
Regarding claim 2,
Lukierski discloses:
further comprising determining a pose of the camera utilizing at least one sensor system of the vehicle and creating the three-dimensional depth map is based on the images from the camera and the pose of the camera.
(See at least Fig. 6 and Para. 0075, wherein a vehicle sensor is used for determining the pose of the camera and therefore creating the 3D depth map based on images from the camera and the pose of the camera.)
Regarding claim 3,
Lukierski discloses:
wherein the at least one sensor system comprises one of at least an accelerometer, a wheel speed sensor, or a wheel angle sensor. (See at least Fig. 6 and Para. 0075, wherein the vehicle sensor comprises a wheel speed sensor.)
Regarding claim 4,
Lukierski discloses:
further comprising a dynamic model of vehicle odometry and determining a pose of the camera utilizing information from the dynamic model.
(See at least Para. 0075, wherein a dynamic model of the vehicle odometry is used to determine a pose of the camera.)
Regarding claim 5,
Lukierski discloses:
wherein the at least one camera mounted on the vehicle comprises at least a front camera, a first side camera and a second side camera. (See at least Fig. 1A and Para. 0039, “the camera device 110 may be statically mounted within a body portion of the test robotic device 105”. The at least one camera is mounted on the vehicle comprises a front camera.)
Regarding claim 6,
Lukierski discloses:
wherein the at least one camera comprises a mono-camera. (See at least Para. 0067, “At block 610, image data is obtained from a monocular multi-directional camera device coupled to the robotic device.” The at least one camera comprises a mono-camera.)
Regarding claim 7,
Lukierski does not specifically state “wherein creating the two-dimensional map further comprises using a pose of the vehicle camera and the two- dimensional laser scan”. However, Kamarulzaman teaches:
wherein creating the two-dimensional map further comprises using a pose of the vehicle camera and the two- dimensional laser scan (See at least Section 3.3.1 “Gmapping” and Equation 10, wherein Gmapping requires both odometry and scan observations. Further see Section 3.3.1. “The main idea of the RBPF is to estimate the trajectory of the robot, x_1:t and the map m, given the observations, z_1:t and the odometry data, u_1:t “ and “The factorization simplifies the computations such that it allows the process to be carried out in two steps. First, the trajectory of the robot can be estimated using the odometry data and the observations. Then, the map p(m|x_1:t, z_1:t-1)can be computed since x_1:t and z_1:t are known.” The 2D map comprises using a pose of the vehicle camera (i.e. the depth camera) and the 2D laser scan. Fig. 6 shows the resultant 2D map.)
It would have been obvious to one of the ordinary skill in the art before the effective filing date of the claimed invention to have modified Lukierski to incorporate the teachings of Kamarulzaman to include the limitations as recited above since “the Kinect has advantages in terms of the extra dimensionality it provides (i.e., three dimensions) and its significantly lower cost” and “This aspect is very important in order to perform SLAM in a real environment where there exist objects of variable shape and size. The robot will also able to avoid certain obstacles that are typically unseen by 2D sensors” (See Section 2. “Kinect vs Laser Scanners”, Kamarulzaman). This would create a more robust system for creating a map of an environment surrounding a vehicle, by adding a step of converting the 3D depth map into a laser scan before creating the 2D map. Additionally, a person having ordinary skill in the art would have a reasonable expectation of success in combining the teachings of Lukierski and Kamarulzaman. The claimed invention is merely a combination of known elements and in combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art before the effective filing date of the claimed invention would have recognized that the results of the combination would have been predictable.
Regarding claim 8,
Lukierski discloses:
wherein the two-dimensional map is created with a local reference coordinate system. (See at least Para. 0062.)
Regarding claim 9,
Lukierski discloses:
creating the two-dimensional map with a controller disposed within the vehicle and saving the map within a memory device associated with the controller.
(See at least Para. 0039-0040, wherein the controlled disposed within the vehicle can create the 2D map, and save the map within a memory device.)
Regarding claim 10,
Lukierski discloses:
further comprising accessing instructions saved in one of the memory device or a computer readable medium that prompt the controller to create the two-dimensional map.
(See at least Para. 0097, wherein instructions are accessed in a computer readable medium that prompts the controller to create the 2D map.)
Regarding claim 11,
Lukierski discloses:
further comprising communicating the two-dimensional map with a vehicle control system. (See at least Para. 0047 and Fig. 4B, which shows the communicating pipeline of the 2D map with a vehicle control system.)
Regarding claim 12,
Lukierski and Kamarulzaman discloses the same limitations as recited in claim 1 above, and is therefore rejected under the same rejection and obviousness rational.
Lukierski further discloses:
An autonomous vehicle system (See at least Para. 0042, wherein a cleaning robot is a robot that autonomously cleans.)
Regarding claim 13,
Lukierski discloses the same limitations as recited in claim 2 above, and therefore the same rejection and obviousness rational applies.
Regarding claim 14,
Lukierski discloses:
wherein the controller is further configured to utilize the pose for the creation of the two-dimensional map. (See at least Fig. 6, wherein the pose is utilized for the creation of the occupancy map (i.e. the 2D map).)
Regarding claim 15,
Lukierski discloses:
wherein the at least one sensor system of the vehicle comprises at least one an accelerometer, a wheel speed sensor, a wheel angle sensor, an inertial measurement unit or a global positioning system. (See at least Para. 0075, wherein a wheel speed sensor is included.)
Regarding claim 16,
Lukierski discloses the same limitations as recited in claim 5 above, and therefore the same rejection and obviousness rational applies.
Regarding claim 17,
Lukierski discloses the same limitations as recited in claim 6 above, and therefore the same rejection and obviousness rational applies.
Regarding claim 18,
Lukierski discloses
further comprising a vehicle control system that utilizes the two-dimensional map to define interaction of the vehicle with the surrounding environment represented by the two-dimensional map. (See at least Para. 0070, wherein a cleaning robot device uses the occupancy map to clean areas on the map.)
Regarding claim 19,
Lukierski and Kamarulzaman discloses the same limitations as recited in claim 1 above, and is therefore rejected under the same rejection and obviousness rational.
Regarding claim 20,
Lukierski discloses the same limitations as recited in claim 2 above, and is therefore rejected under the same rejection and obviousness rational.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GABRIEL J LAMBERT whose telephone number is (571)272-4334. The examiner can normally be reached M-F 10:00 am- 6:00 pm MDT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Erin Piateski can be reached at (571) 270-7429. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Erin M Piateski/Supervisory Patent Examiner, Art Unit 3669
/G.J.L./
Examiner
Art Unit 3669
Prosecution Insights