DETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
2. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 01/08/2026 has been entered.
Status of Claims
3. This office action is in response to application number 18/194,662 filed on 04/03/2023,
and the amendments and arguments filed on 01/08/2026.
Claims 1 and 7-9 have been amended.
No claims have been added.
Claim 4 has been cancelled.
Priority
4. Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C 119 (a)-(d). The certified copy has been filed in parent Application No.JP2022-069149, filed on 04/20/2022.
Information Disclosure Statement
5. The information disclosure statements (IDS) submitted on 04/03/2023, 07/25/2023, 09/29/2023, 04/30/2024, and 08/01/2025 have been received and considered.
Response to Amendment
6. Applicant' s amendments to the Claims have overcome the objection and rejection(s) previously set forth in the Final Office Action mailed 10/08/2025. Applicants arguments, see page 7-9 filed on 01/08/2026, with respect to the rejection(s) of claim(s) 1-10 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. A new grounds for rejection is made under 35 USC 103 as necessitated by amendment over Holz (US 20180306587 A1) in view of Orsulic (WO 2019122939 A1) with regards to claims 1, 5-7, and 9-10. Another grounds for rejection is made under 35 USC 103 as necessitated by amendment over Holz (US 20180306587 A1) in view of Orsulic (WO 2019122939 A1) and further in view of (US 20150310310 A1) to Hesch et al. (hereinafter Hesch) with regards to claims 2-3. Finally with regards to claim 8 a rejection is made under 35 USC 103 as necessitated by amendment over Orsulic (WO 2019122939 A1) in view of Holz (US 20180306587 A1).
Claim Objections
7. Claim 3 objected to because of the following informalities:
Claim 3 reads “the drawing information” but should read “the CAD information”.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
8. Claim(s) 1, 5-7, and 9-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Holz (US 20180306587 A1) in view of Orsulic (WO 2019122939 A1).
Regarding claim 1, Holz discloses An information processing device comprising at least one processor or circuit configured to function as: (Holz Paragraph 0028: “In order to speed up the mapping process and enable real-time user feedback, a computing system may perform a simultaneous localization and mapping (SLAM) process to build a map of an unknown environment (e.g., a warehouse) using measurements provided by a sensor while the sensor provides measurements from different positions within the environment.”) a map acquisition unit configured to acquire a map for measuring a position and an orientation of a movable apparatus, the map being generated by Simultaneous Localization and Mapping (SLAM map); (Holz Paragraph 0098: “A computing system may perform SLAM or similar processes to determine a location of a robot while also detecting positions of markers within the environment. For example, the computing system may detect markers based on the intensity of measurements within laser scans and may match observed markers from different sets of measurements.”) (Holz Paragraph 0100: “The computing system may use control instructions from the robot's control system when estimating its current pose. For instance, odometry data provided from sensors positioned on wheels of the robot may be used to estimate changes in the current pose of the robot. As an example, a robot may have an initial 2D position (x,y) with initial orientation θ.”) (Holz Paragraph 0102: The computing system may perform SLAM iteratively to continuously update the map of markers in the space enabling the robot to map the environment while also navigating safely.”) a detection unit configured to detect a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; (Holz Paragraph 0150: “In some SLAM graphs, a graph may include pose and scans in the nodes and edges formed by matches of marker detections from two poses (nodes). When simultaneously doing alignment, the computing system may add additional edges representing matches of distance measurements (not marker detections) to the closest points sampled from the CAD model. Instead of using a point-to-error, the computing system may use a point to line error to optimize the poses in a way that minimize the distances between matching marker detections and between measured 2D scan point to walls and racking in the CAD model.”) an alignment unit configured to perform alignment of a coordinate system between the SLAM map and the CAD information; (Holz Paragraph 0144: “Design model 1210 represents a 2D overview model of the same environment represented by map of markers 1200, and shows positions of physical structures, such as storage racks, walls, and other features. In the example illustration shown in FIG. 12A, a coordinate frame of map of markers 1200 is not aligned with a coordinate frame of design model 1210. A computing system, however, may determine a transformation that aligns the coordinate frame of map of markers 1200 with the coordinate frame of design model 1210. For instance, the computing system may use measurements of surfaces nearby the robot to determine an occupancy grid map and further determine a transformation that relates occupied cells in the occupancy grid map to sampled points from the design model.”) (Note The design model=CAD and map of markers=SLAM) an association unit configured to perform association between position information of the visual index included in the CAD information after the alignment and position information of groups of feature points corresponding to the visual index included in a constituent element comprising the SLAM map; (Holz Paragraph 0151: “The graph-based optimization problem may involve minimizing a separate edge for each point-to-line normal distance between an occupied cell in the occupancy grid map and a line from the design model that contains a sampled point. The computing system may solve the graph-based optimization problem to align points in the map of markers with points in the design model while simultaneously performing SLAM.”) (Note: In order to determine the alignment error association between the features between CAD and SLAM have to be determined)
Holz does not disclose […] and a correction unit configured to perform a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index.
However, Orsulic does teach […] and a correction unit configured to perform a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index. (Orsulic Page 2, line number 26-30: “Therefore, the invention described herein provides specific advantages over prior art. Appropriate map alignment and correction of defects are achieved by iteratively inputting corrections during the SLAM process. This may also be facilitated by automatically aligning the rangefinder sensor measurements, e.g. from a laser scanner, with for example a reference CAD during the SLAM process, although the user may perform further tuning in case the automatic alignment with the reference CAD produces unsatisfactory results.”) (Orsulic Page 13, line number 11-14: “Figures 10 to 12 illustrate inputting of a pose correction by aligning the incorrectly registered objects 12 such as walls; namely, visually aligning the rewind point cloud 15 with the reference CAD 14, or with the rest of the built environment map 10. The result of alignment is a pose-corrected rewind point cloud 17, as illustrated in Figure 10.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Holz to include […] and a correction unit configured to perform a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index taught by Orsulic. This would have been for the benefit to provide an interactive computer-implemented method and an intuitive, easy- to-use, graphical user interface enabling users without expert knowledge about SLAM to tune the final output, i.e. the built environment map and trajectories, by intervention into the SLAM process, i.e. the execution of the SLAM algorithm in the used SLAM implementation. [Orsulic Page 2, line number 33-Page 3, line number 2]
Regarding claim 5, Holz discloses The information processing device according to claim 1, wherein the map is generated by measuring a surrounding environment of the movable apparatus by a sensor included in the movable apparatus. (Holz Paragraph 0088: “For instance, the robot may use a light ranging and detection (LIDAR) unit that emits light to an area surrounding the robot, and markers positioned in the area surrounding the robot may reflect the light back for detection by a sensor of the robot.”) (Holz Paragraph 0089: “Reflected signals 706 received at sensor 702 may indicate locations of the markers relative to the robot. A computing system may use these determined locations of the markers to develop a map of the markers. In some cases, the computing system may use detected markers to supplement an already generated map of the marker positions. For instance, the computing system may modify the position of one or more markers in the map using new measurements.”)
Regarding claim 6, Holz discloses The information processing device according to claim 1, wherein the map includes a 3-dimensional map used for measuring a position and an orientation of the movable apparatus. (Holz Paragraph 0028: “In order to speed up the mapping process and enable real-time user feedback, a computing system may perform a simultaneous localization and mapping (SLAM) process to build a map of an unknown environment (e.g., a warehouse) using measurements provided by a sensor while the sensor provides measurements from different positions within the environment. SLAM may involve marker detection, data association, pose estimation, and pose/marker refinement, and can be performed in either two-dimensions (2D) or three-dimensions (3D) using a variety of sensor data, such as laser scans of the environment.”) (Holz Paragraph 0090: “A computing system may also determine the pose of the robot as the robot navigates using the map of markers. The computing system may match detected markers with markers in the map to determine the robot's position and orientation.”)
Regarding claim 7, Holz discloses An information processing system comprising at least one processor or circuit configured to function as: (Holz Paragraph 0028: “In order to speed up the mapping process and enable real-time user feedback, a computing system may perform a simultaneous localization and mapping (SLAM) process to build a map of an unknown environment (e.g., a warehouse) using measurements provided by a sensor while the sensor provides measurements from different positions within the environment.”) a map acquisition unit configured to acquire a map for measuring a position and an orientation of a movable apparatus, the map being generated by Simultaneous Localization and Mapping (SLAM map); (Holz Paragraph 0098: “A computing system may perform SLAM or similar processes to determine a location of a robot while also detecting positions of markers within the environment. For example, the computing system may detect markers based on the intensity of measurements within laser scans and may match observed markers from different sets of measurements.”) (Holz Paragraph 0100: “The computing system may use control instructions from the robot's control system when estimating its current pose. For instance, odometry data provided from sensors positioned on wheels of the robot may be used to estimate changes in the current pose of the robot. As an example, a robot may have an initial 2D position (x,y) with initial orientation θ.”) (Holz Paragraph 0102: The computing system may perform SLAM iteratively to continuously update the map of markers in the space enabling the robot to map the environment while also navigating safely.”) a detection unit configured to detect a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; (Holz Paragraph 0150: “In some SLAM graphs, a graph may include pose and scans in the nodes and edges formed by matches of marker detections from two poses (nodes). When simultaneously doing alignment, the computing system may add additional edges representing matches of distance measurements (not marker detections) to the closest points sampled from the CAD model. Instead of using a point-to-error, the computing system may use a point to line error to optimize the poses in a way that minimize the distances between matching marker detections and between measured 2D scan point to walls and racking in the CAD model.”) an alignment unit configured to perform alignment of a coordinate system between the SLAM map and the CAD information; (Holz Paragraph 0144: “Design model 1210 represents a 2D overview model of the same environment represented by map of markers 1200, and shows positions of physical structures, such as storage racks, walls, and other features. In the example illustration shown in FIG. 12A, a coordinate frame of map of markers 1200 is not aligned with a coordinate frame of design model 1210. A computing system, however, may determine a transformation that aligns the coordinate frame of map of markers 1200 with the coordinate frame of design model 1210. For instance, the computing system may use measurements of surfaces nearby the robot to determine an occupancy grid map and further determine a transformation that relates occupied cells in the occupancy grid map to sampled points from the design model.”) (Note The design model=CAD and map of markers=SLAM) an association unit configured to perform association between position information of the visual index included in the CAD information after the alignment and position information of groups of feature points corresponding to the visual index included in a constituent element comprising the SLAM map; (Holz Paragraph 0151: “The graph-based optimization problem may involve minimizing a separate edge for each point-to-line normal distance between an occupied cell in the occupancy grid map and a line from the design model that contains a sampled point. The computing system may solve the graph-based optimization problem to align points in the map of markers with points in the design model while simultaneously performing SLAM.”) (Note: In order to determine the alignment error association between the features between CAD and SLAM have to be determined)
Holz does not disclose […] a correction unit configured to perform a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index.
However, Orsulic does teach […] a correction unit configured to perform a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index. (Orsulic Page 2, line number 26-30: “Therefore, the invention described herein provides specific advantages over prior art. Appropriate map alignment and correction of defects are achieved by iteratively inputting corrections during the SLAM process. This may also be facilitated by automatically aligning the rangefinder sensor measurements, e.g. from a laser scanner, with for example a reference CAD during the SLAM process, although the user may perform further tuning in case the automatic alignment with the reference CAD produces unsatisfactory results.”) (Orsulic Page 13, line number 11-14: “Figures 10 to 12 illustrate inputting of a pose correction by aligning the incorrectly registered objects 12 such as walls; namely, visually aligning the rewind point cloud 15 with the reference CAD 14, or with the rest of the built environment map 10. The result of alignment is a pose-corrected rewind point cloud 17, as illustrated in Figure 10.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Holz to include […] a correction unit configured to perform a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index taught by Orsulic. This would have been for the benefit to provide an interactive computer-implemented method and an intuitive, easy- to-use, graphical user interface enabling users without expert knowledge about SLAM to tune the final output, i.e. the built environment map and trajectories, by intervention into the SLAM process, i.e. the execution of the SLAM algorithm in the used SLAM implementation. [Orsulic Page 2, line number 33-Page 3, line number 2]
Regarding claim 9, Holz discloses A non-transitory computer-readable storage medium configured to store a computer program comprising instructions for executing following processes: (Holz Paragraph 0005: “The system may include a sensor, a computing system, a non-transitory computer readable medium, and program instructions stored on the non-transitory computer readable medium and executable by the computing system to”) acquiring the map that is generated by Simultaneous Localization and Mapping (SLAM map); (Holz Paragraph 0098: “A computing system may perform SLAM or similar processes to determine a location of a robot while also detecting positions of markers within the environment. For example, the computing system may detect markers based on the intensity of measurements within laser scans and may match observed markers from different sets of measurements.”) (Holz Paragraph 0100: “The computing system may use control instructions from the robot's control system when estimating its current pose. For instance, odometry data provided from sensors positioned on wheels of the robot may be used to estimate changes in the current pose of the robot. As an example, a robot may have an initial 2D position (x,y) with initial orientation θ.”) (Holz Paragraph 0102: The computing system may perform SLAM iteratively to continuously update the map of markers in the space enabling the robot to map the environment while also navigating safely.”) detecting a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; (Holz Paragraph 0150: “In some SLAM graphs, a graph may include pose and scans in the nodes and edges formed by matches of marker detections from two poses (nodes). When simultaneously doing alignment, the computing system may add additional edges representing matches of distance measurements (not marker detections) to the closest points sampled from the CAD model. Instead of using a point-to-error, the computing system may use a point to line error to optimize the poses in a way that minimize the distances between matching marker detections and between measured 2D scan point to walls and racking in the CAD model.”) performing alignment of a coordinate system between the SLAM map and the CAD information; (Holz Paragraph 0144: “Design model 1210 represents a 2D overview model of the same environment represented by map of markers 1200, and shows positions of physical structures, such as storage racks, walls, and other features. In the example illustration shown in FIG. 12A, a coordinate frame of map of markers 1200 is not aligned with a coordinate frame of design model 1210. A computing system, however, may determine a transformation that aligns the coordinate frame of map of markers 1200 with the coordinate frame of design model 1210. For instance, the computing system may use measurements of surfaces nearby the robot to determine an occupancy grid map and further determine a transformation that relates occupied cells in the occupancy grid map to sampled points from the design model.”) (Note The design model=CAD and map of markers=SLAM) performing association between position information of the visual index included in the CAD information after the alignment and position information of groups of feature points corresponding to the visual index included in a constituent element comprising the SLAM map; (Holz Paragraph 0151: “The graph-based optimization problem may involve minimizing a separate edge for each point-to-line normal distance between an occupied cell in the occupancy grid map and a line from the design model that contains a sampled point. The computing system may solve the graph-based optimization problem to align points in the map of markers with points in the design model while simultaneously performing SLAM.”) (Note: In order to determine the alignment error association between the features between CAD and SLAM have to be determined)
Holz does not disclose […] and performing a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index.
However, Orsulic does teach […] and performing a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index. (Orsulic Page 2, line number 26-30: “Therefore, the invention described herein provides specific advantages over prior art. Appropriate map alignment and correction of defects are achieved by iteratively inputting corrections during the SLAM process. This may also be facilitated by automatically aligning the rangefinder sensor measurements, e.g. from a laser scanner, with for example a reference CAD during the SLAM process, although the user may perform further tuning in case the automatic alignment with the reference CAD produces unsatisfactory results.”) (Orsulic Page 13, line number 11-14: “Figures 10 to 12 illustrate inputting of a pose correction by aligning the incorrectly registered objects 12 such as walls; namely, visually aligning the rewind point cloud 15 with the reference CAD 14, or with the rest of the built environment map 10. The result of alignment is a pose-corrected rewind point cloud 17, as illustrated in Figure 10.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Holz to include […] and performing a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index taught by Orsulic. This would have been for the benefit to provide an interactive computer-implemented method and an intuitive, easy- to-use, graphical user interface enabling users without expert knowledge about SLAM to tune the final output, i.e. the built environment map and trajectories, by intervention into the SLAM process, i.e. the execution of the SLAM algorithm in the used SLAM implementation. [Orsulic Page 2, line number 33-Page 3, line number 2]
Regarding claim 10, Holz in view of Orsulic teaches claim 1, accordingly, the rejection of claim 1 is incorporated above.
Holz does not disclose The information processing device according to claim 1, wherein the correction unit corrects the map by replacing coordinates of the groups of feature points included in the constituent element for which the association has been made with coordinates of the visual index and fixing the coordinates and by optimizing coordinates of groups of feature points included in another constituent element using results of the fixing.
However, Orsulic does teach The information processing device according to claim 1, wherein the correction unit corrects the map by replacing coordinates of the groups of feature points included in the constituent element for which the association has been made with coordinates of the visual index and fixing the coordinates (Orsulic Page 2, line number 26-30: “Therefore, the invention described herein provides specific advantages over prior art. Appropriate map alignment and correction of defects are achieved by iteratively inputting corrections during the SLAM process. This may also be facilitated by automatically aligning the rangefinder sensor measurements, e.g. from a laser scanner, with for example a reference CAD during the SLAM process, although the user may perform further tuning in case the automatic alignment with the reference CAD produces unsatisfactory results.”) (Orsulic Page 4, line number 2-3: “A 3D pose consists of position, commonly expressed using 3D coordinates (x, y, z ) and orientation, commonly expressed using Euler angles, a rotation matrix, or quaternions.”) (Orsulic Page 13, line number 11-14: “Figures 10 to 12 illustrate inputting of a pose correction by aligning the incorrectly registered objects 12 such as walls; namely, visually aligning the rewind point cloud 15 with the reference CAD 14, or with the rest of the built environment map 10. The result of alignment is a pose-corrected rewind point cloud 17, as illustrated in Figure 10.”) and by optimizing coordinates of groups of feature points included in another constituent element using results of the fixing. (Orsulic Page 4, line number 2-3: “A 3D pose consists of position, commonly expressed using 3D coordinates (x, y, z ) and orientation, commonly expressed using Euler angles, a rotation matrix, or quaternions.”) (Orsulic Page 9, line number 25-31: “The order of insertion of constraints into the pose graph does not matter. A constraint calculated from a user pose correction may be inserted into the pose graph at any time. Afterwards, the optimization process is run to obtain a rebuilt trajectory and map. Thus, graph SLAM is retroactively correctable, easily enabling an iterative workflow without the need to re-execute the complete SLAM process. If the optimization process and rebuilding the environment map is quick enough (which is the case with the preferred embodiment that uses Google Cartographer), the user may quickly see the effect of applying the inputted corrections.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Holz to include The information processing device according to claim 1, wherein the correction unit corrects the map by replacing coordinates of the groups of feature points included in the constituent element for which the association has been made with coordinates of the visual index and fixing the coordinates and by optimizing coordinates of groups of feature points included in another constituent element using results of the fixing taught by Orsulic. This would have been for the benefit to provide an interactive computer-implemented method and an intuitive, easy- to-use, graphical user interface enabling users without expert knowledge about SLAM to tune the final output, i.e. the built environment map and trajectories, by intervention into the SLAM process, i.e. the execution of the SLAM algorithm in the used SLAM implementation. [Orsulic Page 2, line number 33-Page 3, line number 2]
9. Claim(s) 2-3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Holz (US 20180306587 A1) in view of Orsulic (WO 2019122939 A1) and further in view of (US 20150310310 A1) to Hesch et al. (hereinafter Hesch).
Regarding claim 2, Holz in view of Orsulic teaches claim 1, accordingly, the rejection of claim 1 is incorporated above.
Holz in view of Orsulic does not teach The information processing device according to claim 1, wherein the visual index includes a feature point or an edge.
However, Hesch does teach The information processing device according to claim 1, wherein the visual index includes a feature point or an edge. (Hesch Paragraph 0017: “FIG. 1 illustrates an electronic device 100 configured to support location-based functionality, such as SLAM or AR, using image and non-image sensor data in accordance with at least one embodiment of the present disclosure.”) (Hesch Paragraph 0026: “From this input data, the electronic device 100 can determine its relative pose without explicit absolute localization information from an external source. To illustrate, the electronic device 100 can perform analysis of the wide angle imaging camera image data 134 and the narrow angle imaging camera image data 136 to determine the distances between the electronic device 100 and the corners 124, 126, 128.”) (Hesch Paragraph 0045: “The datastores further can include a SLAM/AR datastore 442 that stores SLAM-based information, such as mapping information for areas of the local environment 112 (FIG. 1) already explored by the electronic device 100, or AR information, such as CAD-based representations of the relative locations of objects of interest in the local environment 112.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Holz in view of Orsulic to include The information processing device according to claim 1, wherein the visual index includes a feature point or an edge taught by Hesch. This would have been for the benefit to provide an enhanced technique of determining a relative position or relative orientation of an electronic device based on image-based identification of objects in a local environment of the electronic device. [Hesch Paragraph 0013]
Regarding claim 3, Holz in view of Orsulic teaches claim 1, accordingly, the rejection of claim 1 is incorporated above.
Holz in view of Orsulic does not teach The information processing device according to claim 1, wherein the detection unit generates an image associated with an arbitrary key frame of the map based on the drawing information and detects the visual index from the image.
However, Hesch does teach The information processing device according to claim 1, wherein the detection unit generates an image associated with an arbitrary key frame of the map based on the drawing information and detects the visual index from the image. (Hesch Paragraph 0005: “FIG. 2 is a diagram illustrating adjustment of a refined pose based on pose tracking of the electronic device of FIG. 1 in a free frame of reference in accordance with at least one embodiment of the present disclosure.”)
PNG
media_image1.png
459
379
media_image1.png
Greyscale
(Hesch Paragraph 0014: “FIGS. 1-9 illustrate various techniques for the determination of a pose of an electronic device within a local environment so as to support location-based functionality, such as augmented reality (AR) functionality, visual odometry or other simultaneous localization and mapping (SLAM) functionality, and the like. The term “pose” is used herein to refer to either or both of position (also referred to as a location) and orientation (also referred to as a point of view).”) (Hesch Paragraph 0016: “Concurrent with matching the generated set of descriptors, the electronic device can track changes in its pose (e.g. changes in its location, point of view, or both) in an arbitrary, or “free”, frame of reference (sometimes referred to herein as “free space”).”) (Hesch Paragraph 0026: “From this input data, the electronic device 100 can determine its relative pose without explicit absolute localization information from an external source. To illustrate, the electronic device 100 can perform analysis of the wide angle imaging camera image data 134 and the narrow angle imaging camera image data 136 to determine the distances between the electronic device 100 and the corners 124, 126, 128.”) (Hesch Paragraph 0045: “The datastores further can include a SLAM/AR datastore 442 that stores SLAM-based information, such as mapping information for areas of the local environment 112 (FIG. 1) already explored by the electronic device 100, or AR information, such as CAD-based representations of the relative locations of objects of interest in the local environment 112.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Holz in view of Orsulic to include The information processing device according to claim 1, wherein the detection unit generates an image associated with an arbitrary key frame of the map based on the drawing information and detects the visual index from the image taught by Hesch. This would have been for the benefit to provide an enhanced technique of determining a relative position or relative orientation of an electronic device based on image-based identification of objects in a local environment of the electronic device. [Hesch Paragraph 0013]
10. Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Orsulic (WO 2019122939 A1) in view of Holz (US 20180306587 A1).
Regarding claim 8, Orsulic discloses A method of controlling an information processing device that corrects a map for measuring a position and an orientation of a movable apparatus, the method comprising: (Orsulic Page 3, line number 3-line number 5: “Another object of the present invention is to provide an interactive computer-implemented method by which the environment map building process i.e. the SLAM process is not restricted to data collected from a single mobile device.”) (Orsulic Page 3, line number 11-12: “Further, the present method allows for assisted inputting of user corrections by means of scan matching, making initial inputting of user corrections fast and accurate”) […] and performing a correction that replaces the position information of groups of feature points included in the constituent element for which the association has been made with the position information of the visual index. (Orsulic Page 2, line number 26-30: “Therefore, the invention described herein provides specific advantages over prior art. Appropriate map alignment and correction of defects are achieved by iteratively inputting corrections during the SLAM process. This may also be facilitated by automatically aligning the rangefinder sensor measurements, e.g. from a laser scanner, with for example a reference CAD during the SLAM process, although the user may perform further tuning in case the automatic alignment with the reference CAD produces unsatisfactory results.”) (Orsulic Page 13, line number 11-14: “Figures 10 to 12 illustrate inputting of a pose correction by aligning the incorrectly registered objects 12 such as walls; namely, visually aligning the rewind point cloud 15 with the reference CAD 14, or with the rest of the built environment map 10. The result of alignment is a pose-corrected rewind point cloud 17, as illustrated in Figure 10.”)
Orsulic does not disclose […] acquiring the map that is generated by Simultaneous Localization and Mapping (SLAM map); detecting a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; performing alignment of a coordinate system between the SLAM map and the CAD information; performing association between position information of the visual index included in the CAD information after the alignment and position information of groups of feature points corresponding to the visual index included in a constituent element comprising the SLAM map;
However, Holz does teach […] acquiring the map that is generated by Simultaneous Localization and Mapping (SLAM map); (Holz Paragraph 0098: “A computing system may perform SLAM or similar processes to determine a location of a robot while also detecting positions of markers within the environment. For example, the computing system may detect markers based on the intensity of measurements within laser scans and may match observed markers from different sets of measurements.”) (Holz Paragraph 0100: “The computing system may use control instructions from the robot's control system when estimating its current pose. For instance, odometry data provided from sensors positioned on wheels of the robot may be used to estimate changes in the current pose of the robot. As an example, a robot may have an initial 2D position (x,y) with initial orientation θ.”) (Holz Paragraph 0102: The computing system may perform SLAM iteratively to continuously update the map of markers in the space enabling the robot to map the environment while also navigating safely.”) detecting a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; (Holz Paragraph 0150: “In some SLAM graphs, a graph may include pose and scans in the nodes and edges formed by matches of marker detections from two poses (nodes). When simultaneously doing alignment, the computing system may add additional edges representing matches of distance measurements (not marker detections) to the closest points sampled from the CAD model. Instead of using a point-to-error, the computing system may use a point to line error to optimize the poses in a way that minimize the distances between matching marker detections and between measured 2D scan point to walls and racking in the CAD model.”) performing alignment of a coordinate system between the SLAM map and the CAD information; (Holz Paragraph 0144: “Design model 1210 represents a 2D overview model of the same environment represented by map of markers 1200, and shows positions of physical structures, such as storage racks, walls, and other features. In the example illustration shown in FIG. 12A, a coordinate frame of map of markers 1200 is not aligned with a coordinate frame of design model 1210. A computing system, however, may determine a transformation that aligns the coordinate frame of map of markers 1200 with the coordinate frame of design model 1210. For instance, the computing system may use measurements of surfaces nearby the robot to determine an occupancy grid map and further determine a transformation that relates occupied cells in the occupancy grid map to sampled points from the design model.”) (Note The design model=CAD and map of markers=SLAM) performing association between position information of the visual index included in the CAD information after the alignment and position information of groups of feature points corresponding to the visual index included in a constituent element comprising the SLAM map; (Holz Paragraph 0151: “The graph-based optimization problem may involve minimizing a separate edge for each point-to-line normal distance between an occupied cell in the occupancy grid map and a line from the design model that contains a sampled point. The computing system may solve the graph-based optimization problem to align points in the map of markers with points in the design model while simultaneously performing SLAM.”) (Note: In order to determine the alignment error association between the features between CAD and SLAM have to be determined)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Orsulic to include […] acquiring the map that is generated by Simultaneous Localization and Mapping (SLAM map); detecting a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; performing alignment of a coordinate system between the SLAM map and the CAD information; performing association between position information of the visual index included in the CAD information after the alignment and position information of groups of feature points corresponding to the visual index included in a constituent element comprising the SLAM map; taught by Holz. This would have been for the benefit to provide a better system which include means for determining a transformation between the map of the plurality of markers and a design model of the environment that relates occupied cells in the occupancy grid map to sampled points from the design model, and means for providing the transformation between the map of the plurality of markers and the design model. [Orsulic Paragraph 0007]
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN J HARVEY whose telephone number is 571-272-5327. The examiner can normally be reached 8:00AM-5:00PM M-Th, 8:00AM-4:00PM F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kito Robinson can be reached at 571-270-3921. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.J.H./Junior Patent Examiner, Art Unit 3664
/KITO R ROBINSON/Supervisory Patent Examiner, Art Unit 3664