PDETAILED ACTION
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
2. This office action is in response to application number 18/194,662 filed on 04/03/2023,
and the amendments and arguments filed on 06/11/2026.
Claims 1, 3, and 7-9 have been amended.
Claims 11 and 12 have been added.
Claim 4 has been cancelled.
Priority
3. Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C 119 (a)-(d). The certified copy has been filed in parent Application No.JP2022-069149, filed on 04/20/2022.
Information Disclosure Statement
4. The information disclosure statements (IDS) submitted on 04/03/2023, 07/25/2023, 09/29/2023, 04/30/2024, and 08/01/2025 have been received and considered.
Response to Amendment
5. Applicant' s amendments to the Claims have overcome the objection and rejection(s) previously set forth in the Non-Final Office Action mailed 03/11/2026. Applicants arguments, see page 7-11 filed on 06/11/2026, with respect to the rejection(s) of claim(s) 1-3 and 5-10 under 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. A new grounds for rejection is made under 35 USC 112(b). Additionally, another new grounds for rejection is made under 35 USC 103 as necessitated by amendment over Abel (DE 102021130631 A1) in view of Tomioka (WO 2019138834 A1) further in view of Chan (US 20210348941 A1) further in view of Hesch (US 20150310310 A1) and further in view of Jing (CN 113124850 A).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
6. Claim 1, 7, and 10 rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation " after the registration alignment" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation " after the registration alignment" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Claim 10 recites the limitation "in the constituent element…in another constituent element using results of the fixing." in line 2-3. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
7. Claim(s) 1-2 and 5-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over
Abel (DE 102021130631 A1) in view of Tomioka (WO 2019138834 A1) and further in view of (US 20210348941 A1) to Chan et al. (hereinafter Chan).
Regarding claim 1, Abel discloses An information processing device comprising at least one processor or circuit configured to function as: (Abel Page 2, Paragraph 1: “The invention relates to a method for navigating an industrial truck, in particular an autonomous or automated industrial truck, with a sensor system for detecting information in the operating area of the industrial truck and a data processing device for evaluating this information and creating navigation instructions and/or control commands for navigating the industrial truck, with the starting point from a reconnaissance trip, a map of the surroundings is first created in the data processing device in a vehicle coordinate system with the help of the sensors and during a subsequent navigation trip, a scene currently recorded by the sensors is compared in the data processing device with the map of the surroundings and the industrial truck is localized in the vehicle coordinate system.”) a map acquisition unit configured to acquire a map for measuring a position and an orientation of a movable apparatus, the map being generated by Simultaneous Localization and Mapping (SLAM map); (Abel Page 2, Paragraph 4: “Methods have also already been proposed with which an autonomous or automated industrial truck can locate itself in unfamiliar surroundings. This includes the so-called SLAM method, which has been used in robotics for several years.”) (Abel Page 2, Paragraph 5: “The SLAM method started with laser sensors and was then expanded over the years to include other sensors, such as 2D and 3D cameras, inertial measurement units (IMUs), ultrasonic sensors, etc.”) (Abel Page 4, Paragraph 6: “For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device.”) a detection unit configured to detect a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; (Abel Page 3, Paragraph 2: “and the map of the surroundings created in the vehicle coordinate system is compared in the data processing device with the Reference coordinate system is compared, and the localization of the truck in the vehicle coordinate system in the data processing device is converted into a localization of the truck in the reference coordinate system.”) (Abel Page 5, Paragraph 6: “The markings 6 and 7 are wall surfaces of a warehouse, the global positions of which are known in the reference coordinate system 3 and which are detected by the sensors 5 of the industrial truck 1.”) (Note: Reference coordinate system = CAD information) (Note: Vehicle coordinate system = SLAM map) an alignment unit configured to perform alignment of a coordinate system between the SLAM map and the CAD information; (Abel Page 4, Paragraph 6:“For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device. A CAD map of the operating area defining the reference coordinate system is also loaded into the computer program. In the computer program, the map of the environment created by the sensors is placed over the CAD map and the map of the environment is compared with the CAD map of the operating area in such a way that the map of the environment is congruent with the CAD map. This comparison can be carried out automatically by the computer program by optimizing it so that the cards fit together as well as possible.”)
Abel does not disclose […] an association unit configured to perform association between position information of the visual index included in the CAD information after the registration alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; and a correction unit configured to perform a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information.
However, Tomioka does teach […] an association unit configured to perform association between position information of the visual index included in the CAD information after the registration alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; (Tomioka Page 6, Paragraph 7: “Specifically, the control value is calculated such that the Euclidean distance between the destination coordinates held by the holding unit 1130 and the position and orientation of the imaging unit 110 calculated by the calculation unit 1120 is reduced. The control value calculated by the control unit 1140 is output to the actuator 120.”) (Tomioka Page 9, Paragraph 4: “For example, the holding unit 1130 may hold and hold a CAD drawing or map image of the environment as it is or after converting the data format.”) (Tomioka Page 15, Paragraph 2: “In the present embodiment, in particular, the AGV loads and carries a load, and when reaching the destination, the case where it must be strictly stopped at a predetermined position with respect to the shelf and the belt conveyor will be described. In the present embodiment, a method of controlling the AGV by calculating the exact position and orientation by calculating the position and orientation of an object such as a shelf or a belt conveyor imaged by the imaging unit 110 will be described.”) (Tomioka Page 25, Paragraph 2: “In step S810, the position and orientation calculation unit 8110 calculates the position and orientation of the imaging device 110, and creates a three-dimensional map. This is realized by an SLAM (Simultaneous Localization and Mapping) algorithm that performs position and orientation estimation while creating a map based on the position and orientation.”) (Note: In order to determine the control value of the robot, the CAD data and SLAM data corresponding to the same object must be associated.)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel to include […] an association unit configured to perform association between position information of the visual index included in the CAD information after the registration alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; taught by Tomioka. This would have been for the benefit to provide a mor efficient In the present embodiment, the map information is not limited to generation from data acquired by the imaging unit 110 mounted on the mobile object 11. For example, the holding unit 1130 may hold and hold a CAD drawing or map image of the environment as it is or after converting the data format. Alternatively, the holding unit 1130 may hold a map based on a CAD drawing or a map image as an initial map, and update the map using the above-described SLAM technology. The control unit 1140 may calculate the control value for controlling the AGV so as to update the map at a point where a predetermined time has passed, while holding the map update time. The map may be updated by overwriting, or the initial map may be held and the difference may be stored as update information. At this time, the map can be managed in layers and checked on the display unit H16 or can be returned to the initial map. [Tomioka Page 9, Paragraph 4]
Tomioka does not teach […] and a correction unit configured to perform a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information.
However, Chan does teach […] and a correction unit configured to perform a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information. (Chan Paragraph 0009: “In some embodiments, the step of updating the positioning information of the mobile vehicle in the new SLAM map further comprises: using a position and azimuth of the mobile vehicle at the current point to update the SLAM map to the new SLAM map and updating the positioning information, wherein the position and azimuth of the mobile vehicle are detected by the non-SLAM positioning device.”) (Note: Non-slam position = CAD information) (Chan Paragraph 0037: “In step S310, the mobile vehicle detects a first position trajectory and a first azimuth trajectory of the mobile vehicle on the SLAM map by a non-SLAM positioning device. Specifically, the non-SLAM positioning device can instantly record the position and azimuth of the mobile vehicle on the SLAM map.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel in view of Tomioka to include […] and a correction unit configured to perform a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information taught by Chan. This would have been for the benefit to provide a method for relocating a mobile vehicle in a SLAM map and a mobile vehicle to solve the problem of the mobile robot may not be able to relocate itself in the currently drawn SLAM map (i.e., the mobile robot gets lost). [Chan Paragraph 0004 and 0005]
Regarding claim 2, Abel discloses The information processing device according to claim 1, wherein the visual index includes a feature point or an edge. (Abel Page 3, Paragraph 2: “and the map of the surroundings created in the vehicle coordinate system is compared in the data processing device with the Reference coordinate system is compared, and the localization of the truck in the vehicle coordinate system in the data processing device is converted into a localization of the truck in the reference coordinate system.”) (Abel Page 5, Paragraph 6: “In the 2 the use of natural markings 6, 7, 8 for defining the reference coordinate system 3 is shown. The markings 6 and 7 are wall surfaces of a warehouse, the global positions of which are known in the reference coordinate system 3 and which are detected by the sensors 5 of the industrial truck 1. The marking 8 is a sign, for example an emergency exit sign, which is set up at a known global position of the reference coordinate system 3 and is also detected by the sensor system 5 .”)
Regarding claim 5, Abel discloses The information processing device according to claim 1, wherein the map is generated by measuring a surrounding environment of the movable apparatus by a sensor included in the movable apparatus. (Abel Page 2, Paragraph 4: “Methods have also already been proposed with which an autonomous or automated industrial truck can locate itself in unfamiliar surroundings. This includes the so-called SLAM method, which has been used in robotics for several years.”) (Abel Page 2, Paragraph 5: “The SLAM method started with laser sensors and was then expanded over the years to include other sensors, such as 2D and 3D cameras, inertial measurement units (IMUs), ultrasonic sensors, etc.”) (Abel Page 5, Paragraph 5: “a map of the surroundings of the operating area is created in the data processing device 9 of the industrial truck 1 using the sensors 5 of the industrial truck 1”)
PNG
media_image1.png
306
377
media_image1.png
Greyscale
Regarding claim 6, Abel discloses The information processing device according to claim 1, wherein the map includes a 3-dimensional map used for measuring a position and an orientation of the movable apparatus. (Abel Page 2, Paragraph 5: “The SLAM method started with laser sensors and was then expanded over the years to include other sensors, such as 2D and 3D cameras, inertial measurement units (IMUs), ultrasonic sensors, etc.”) (Abel Page 2, Paragraph 6: “The disadvantage of the SLAM method is that when the autonomous or automated industrial truck is initialized, the current vehicle position is assumed to be the origin, i.e. with a pose in relation to the spatial axes x = y = z = 0”)
Regarding claim 7, Abel discloses An information processing system comprising at least one processor or circuit configured to function as: (Abel Page 2, Paragraph 1: “The invention relates to a method for navigating an industrial truck, in particular an autonomous or automated industrial truck, with a sensor system for detecting information in the operating area of the industrial truck and a data processing device for evaluating this information and creating navigation instructions and/or control commands for navigating the industrial truck, with the starting point from a reconnaissance trip, a map of the surroundings is first created in the data processing device in a vehicle coordinate system with the help of the sensors and during a subsequent navigation trip, a scene currently recorded by the sensors is compared in the data processing device with the map of the surroundings and the industrial truck is localized in the vehicle coordinate system.”) (Abel Page 2, Paragraph 2: “The invention also relates to a system for carrying out the method.”) a map acquisition unit configured to acquire a map for measuring a position and an orientation of a movable apparatus, the map being generated by Simultaneous Localization and Mapping (SLAM map); (Abel Page 2, Paragraph 4: “Methods have also already been proposed with which an autonomous or automated industrial truck can locate itself in unfamiliar surroundings. This includes the so-called SLAM method, which has been used in robotics for several years.”) (Abel Page 2, Paragraph 5: “The SLAM method started with laser sensors and was then expanded over the years to include other sensors, such as 2D and 3D cameras, inertial measurement units (IMUs), ultrasonic sensors, etc.”) (Abel Page 4, Paragraph 6: “For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device.”) a detection unit configured to detect a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; (Abel Page 3, Paragraph 2: “and the map of the surroundings created in the vehicle coordinate system is compared in the data processing device with the Reference coordinate system is compared, and the localization of the truck in the vehicle coordinate system in the data processing device is converted into a localization of the truck in the reference coordinate system.”) (Abel Page 5, Paragraph 6: “The markings 6 and 7 are wall surfaces of a warehouse, the global positions of which are known in the reference coordinate system 3 and which are detected by the sensors 5 of the industrial truck 1.”) (Note: Reference coordinate system = CAD information) (Note: Vehicle coordinate system = SLAM map) an alignment unit configured to perform alignment of a coordinate system between the SLAM map and the CAD information; (Abel Page 4, Paragraph 6:“For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device. A CAD map of the operating area defining the reference coordinate system is also loaded into the computer program. In the computer program, the map of the environment created by the sensors is placed over the CAD map and the map of the environment is compared with the CAD map of the operating area in such a way that the map of the environment is congruent with the CAD map. This comparison can be carried out automatically by the computer program by optimizing it so that the cards fit together as well as possible.”)
Abel does not disclose […] an association unit configured to perform association between position information of the visual index included in the CAD information after the registration alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure e as the visual index; and a correction unit configured to perform a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information.
However, Tomioka does teach […] an association unit configured to perform association between position information of the visual index included in the CAD information after the registration alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; (Tomioka Page 6, Paragraph 7: “Specifically, the control value is calculated such that the Euclidean distance between the destination coordinates held by the holding unit 1130 and the position and orientation of the imaging unit 110 calculated by the calculation unit 1120 is reduced. The control value calculated by the control unit 1140 is output to the actuator 120.”) (Tomioka Page 9, Paragraph 4: “For example, the holding unit 1130 may hold and hold a CAD drawing or map image of the environment as it is or after converting the data format.”) (Tomioka Page 15, Paragraph 2: “In the present embodiment, in particular, the AGV loads and carries a load, and when reaching the destination, the case where it must be strictly stopped at a predetermined position with respect to the shelf and the belt conveyor will be described. In the present embodiment, a method of controlling the AGV by calculating the exact position and orientation by calculating the position and orientation of an object such as a shelf or a belt conveyor imaged by the imaging unit 110 will be described.”) (Tomioka Page 25, Paragraph 2: “In step S810, the position and orientation calculation unit 8110 calculates the position and orientation of the imaging device 110, and creates a three-dimensional map. This is realized by an SLAM (Simultaneous Localization and Mapping) algorithm that performs position and orientation estimation while creating a map based on the position and orientation.”) (Note: In order to determine the control value of the robot, the CAD data and SLAM data corresponding to the same object must be associated.)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel to include […] an association unit configured to perform association between position information of the visual index included in the CAD information after the registration alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure e as the visual index; taught by Tomioka. This would have been for the benefit to provide a mor efficient In the present embodiment, the map information is not limited to generation from data acquired by the imaging unit 110 mounted on the mobile object 11. For example, the holding unit 1130 may hold and hold a CAD drawing or map image of the environment as it is or after converting the data format. Alternatively, the holding unit 1130 may hold a map based on a CAD drawing or a map image as an initial map, and update the map using the above-described SLAM technology. The control unit 1140 may calculate the control value for controlling the AGV so as to update the map at a point where a predetermined time has passed, while holding the map update time. The map may be updated by overwriting, or the initial map may be held and the difference may be stored as update information. At this time, the map can be managed in layers and checked on the display unit H16 or can be returned to the initial map. [Tomioka Page 9, Paragraph 4]
Tomioka does not teach […] and a correction unit configured to perform a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information.
However, Chan does teach […] and a correction unit configured to perform a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information. (Chan Paragraph 0009: “In some embodiments, the step of updating the positioning information of the mobile vehicle in the new SLAM map further comprises: using a position and azimuth of the mobile vehicle at the current point to update the SLAM map to the new SLAM map and updating the positioning information, wherein the position and azimuth of the mobile vehicle are detected by the non-SLAM positioning device.”) (Note: Non-slam position = CAD information) (Chan Paragraph 0037: “In step S310, the mobile vehicle detects a first position trajectory and a first azimuth trajectory of the mobile vehicle on the SLAM map by a non-SLAM positioning device. Specifically, the non-SLAM positioning device can instantly record the position and azimuth of the mobile vehicle on the SLAM map.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel in view of Tomioka to include […] and a correction unit configured to perform a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information taught by Chan. This would have been for the benefit to provide a method for relocating a mobile vehicle in a SLAM map and a mobile vehicle to solve the problem of the mobile robot may not be able to relocate itself in the currently drawn SLAM map (i.e., the mobile robot gets lost). [Chan Paragraph 0004 and 0005]
Regarding claim 8, Abel discloses A method of controlling an information processing device that corrects a map for measuring a position and an orientation of a movable apparatus, the method comprising: (Abel Page 2, Paragraph 1: “The invention relates to a method for navigating an industrial truck, in particular an autonomous or automated industrial truck, with a sensor system for detecting information in the operating area of the industrial truck and a data processing device for evaluating this information and creating navigation instructions and/or control commands for navigating the industrial truck, with the starting point from a reconnaissance trip, a map of the surroundings is first created in the data processing device in a vehicle coordinate system with the help of the sensors and during a subsequent navigation trip, a scene currently recorded by the sensors is compared in the data processing device with the map of the surroundings and the industrial truck is localized in the vehicle coordinate system.”) acquiring the map that is generated by Simultaneous Localization and Mapping (SLAM map); (Abel Page 2, Paragraph 4: “Methods have also already been proposed with which an autonomous or automated industrial truck can locate itself in unfamiliar surroundings. This includes the so-called SLAM method, which has been used in robotics for several years.”) (Abel Page 2, Paragraph 5: “The SLAM method started with laser sensors and was then expanded over the years to include other sensors, such as 2D and 3D cameras, inertial measurement units (IMUs), ultrasonic sensors, etc.”) (Abel Page 4, Paragraph 6: “For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device.”) detecting a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; (Abel Page 3, Paragraph 2: “and the map of the surroundings created in the vehicle coordinate system is compared in the data processing device with the Reference coordinate system is compared, and the localization of the truck in the vehicle coordinate system in the data processing device is converted into a localization of the truck in the reference coordinate system.”) (Abel Page 5, Paragraph 6: “The markings 6 and 7 are wall surfaces of a warehouse, the global positions of which are known in the reference coordinate system 3 and which are detected by the sensors 5 of the industrial truck 1.”) (Note: Reference coordinate system = CAD information) (Note: Vehicle coordinate system = SLAM map) performing alignment of a coordinate system between the SLAM map and the CAD information; (Abel Page 4, Paragraph 6:“For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device. A CAD map of the operating area defining the reference coordinate system is also loaded into the computer program. In the computer program, the map of the environment created by the sensors is placed over the CAD map and the map of the environment is compared with the CAD map of the operating area in such a way that the map of the environment is congruent with the CAD map. This comparison can be carried out automatically by the computer program by optimizing it so that the cards fit together as well as possible.”)
Abel does not disclose […] performing association between position information of the visual index included in the CAD information after the alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; and performing a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information.
However, Tomioka does teach […] performing association between position information of the visual index included in the CAD information after the alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; (Tomioka Page 6, Paragraph 7: “Specifically, the control value is calculated such that the Euclidean distance between the destination coordinates held by the holding unit 1130 and the position and orientation of the imaging unit 110 calculated by the calculation unit 1120 is reduced. The control value calculated by the control unit 1140 is output to the actuator 120.”) (Tomioka Page 9, Paragraph 4: “For example, the holding unit 1130 may hold and hold a CAD drawing or map image of the environment as it is or after converting the data format.”) (Tomioka Page 15, Paragraph 2: “In the present embodiment, in particular, the AGV loads and carries a load, and when reaching the destination, the case where it must be strictly stopped at a predetermined position with respect to the shelf and the belt conveyor will be described. In the present embodiment, a method of controlling the AGV by calculating the exact position and orientation by calculating the position and orientation of an object such as a shelf or a belt conveyor imaged by the imaging unit 110 will be described.”) (Tomioka Page 25, Paragraph 2: “In step S810, the position and orientation calculation unit 8110 calculates the position and orientation of the imaging device 110, and creates a three-dimensional map. This is realized by an SLAM (Simultaneous Localization and Mapping) algorithm that performs position and orientation estimation while creating a map based on the position and orientation.”) (Note: In order to determine the control value of the robot, the CAD data and SLAM data corresponding to the same object must be associated.)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel to include […] performing association between position information of the visual index included in the CAD information after the alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; taught by Tomioka. This would have been for the benefit to provide a mor efficient In the present embodiment, the map information is not limited to generation from data acquired by the imaging unit 110 mounted on the mobile object 11. For example, the holding unit 1130 may hold and hold a CAD drawing or map image of the environment as it is or after converting the data format. Alternatively, the holding unit 1130 may hold a map based on a CAD drawing or a map image as an initial map, and update the map using the above-described SLAM technology. The control unit 1140 may calculate the control value for controlling the AGV so as to update the map at a point where a predetermined time has passed, while holding the map update time. The map may be updated by overwriting, or the initial map may be held and the difference may be stored as update information. At this time, the map can be managed in layers and checked on the display unit H16 or can be returned to the initial map. [Tomioka Page 9, Paragraph 4]
Tomioka does not teach […] and performing a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information.
However, Chan does teach […] and performing a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information. (Chan Paragraph 0009: “In some embodiments, the step of updating the positioning information of the mobile vehicle in the new SLAM map further comprises: using a position and azimuth of the mobile vehicle at the current point to update the SLAM map to the new SLAM map and updating the positioning information, wherein the position and azimuth of the mobile vehicle are detected by the non-SLAM positioning device.”) (Note: Non-slam position = CAD information) (Chan Paragraph 0037: “In step S310, the mobile vehicle detects a first position trajectory and a first azimuth trajectory of the mobile vehicle on the SLAM map by a non-SLAM positioning device. Specifically, the non-SLAM positioning device can instantly record the position and azimuth of the mobile vehicle on the SLAM map.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel in view of Tomioka to include […] and performing a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information taught by Chan. This would have been for the benefit to provide a method for relocating a mobile vehicle in a SLAM map and a mobile vehicle to solve the problem of the mobile robot may not be able to relocate itself in the currently drawn SLAM map (i.e., the mobile robot gets lost). [Chan Paragraph 0004 and 0005]
Regarding claim 9, Abel discloses A non-transitory computer-readable storage medium configured to store a computer program comprising instructions for executing following processes: (Abel Page 4, Paragraph 6: “For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device.”) acquiring the map that is generated by Simultaneous Localization and Mapping (SLAM map); (Abel Page 2, Paragraph 4: “Methods have also already been proposed with which an autonomous or automated industrial truck can locate itself in unfamiliar surroundings. This includes the so-called SLAM method, which has been used in robotics for several years.”) (Abel Page 2, Paragraph 5: “The SLAM method started with laser sensors and was then expanded over the years to include other sensors, such as 2D and 3D cameras, inertial measurement units (IMUs), ultrasonic sensors, etc.”) (Abel Page 4, Paragraph 6: “For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device.”) detecting a visual index from CAD information that corresponds to a real space that the movable apparatus moves or an object in the real space, the real space corresponding to the SLAM map; (Abel Page 3, Paragraph 2: “and the map of the surroundings created in the vehicle coordinate system is compared in the data processing device with the Reference coordinate system is compared, and the localization of the truck in the vehicle coordinate system in the data processing device is converted into a localization of the truck in the reference coordinate system.”) (Abel Page 5, Paragraph 6: “The markings 6 and 7 are wall surfaces of a warehouse, the global positions of which are known in the reference coordinate system 3 and which are detected by the sensors 5 of the industrial truck 1.”) (Note: Reference coordinate system = CAD information) (Note: Vehicle coordinate system = SLAM map) performing alignment of a coordinate system between the SLAM map and the CAD information; (Abel Page 4, Paragraph 6:“For this purpose, the environment map in the vehicle coordinate system created by means of the sensor system, for example using the SLAM method, can be exported in particular to a computer program which preferably runs on a data processing device. A CAD map of the operating area defining the reference coordinate system is also loaded into the computer program. In the computer program, the map of the environment created by the sensors is placed over the CAD map and the map of the environment is compared with the CAD map of the operating area in such a way that the map of the environment is congruent with the CAD map. This comparison can be carried out automatically by the computer program by optimizing it so that the cards fit together as well as possible.”)
Abel does not disclose […] performing association between position information of the visual index included in the CAD information after the alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; and performing a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information.
However, Tomioka does teach […] performing association between position information of the visual index included in the CAD information after the alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; (Tomioka Page 6, Paragraph 7: “Specifically, the control value is calculated such that the Euclidean distance between the destination coordinates held by the holding unit 1130 and the position and orientation of the imaging unit 110 calculated by the calculation unit 1120 is reduced. The control value calculated by the control unit 1140 is output to the actuator 120.”) (Tomioka Page 9, Paragraph 4: “For example, the holding unit 1130 may hold and hold a CAD drawing or map image of the environment as it is or after converting the data format.”) (Tomioka Page 15, Paragraph 2: “In the present embodiment, in particular, the AGV loads and carries a load, and when reaching the destination, the case where it must be strictly stopped at a predetermined position with respect to the shelf and the belt conveyor will be described. In the present embodiment, a method of controlling the AGV by calculating the exact position and orientation by calculating the position and orientation of an object such as a shelf or a belt conveyor imaged by the imaging unit 110 will be described.”) (Tomioka Page 25, Paragraph 2: “In step S810, the position and orientation calculation unit 8110 calculates the position and orientation of the imaging device 110, and creates a three-dimensional map. This is realized by an SLAM (Simultaneous Localization and Mapping) algorithm that performs position and orientation estimation while creating a map based on the position and orientation.”) (Note: In order to determine the control value of the robot, the CAD data and SLAM data corresponding to the same object must be associated.)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel to include […] performing association between position information of the visual index included in the CAD information after the alignment and position information of a group of feature points included in the SLAM map and corresponding to the same object or structure as the visual index; taught by Tomioka. This would have been for the benefit to provide a mor efficient In the present embodiment, the map information is not limited to generation from data acquired by the imaging unit 110 mounted on the mobile object 11. For example, the holding unit 1130 may hold and hold a CAD drawing or map image of the environment as it is or after converting the data format. Alternatively, the holding unit 1130 may hold a map based on a CAD drawing or a map image as an initial map, and update the map using the above-described SLAM technology. The control unit 1140 may calculate the control value for controlling the AGV so as to update the map at a point where a predetermined time has passed, while holding the map update time. The map may be updated by overwriting, or the initial map may be held and the difference may be stored as update information. At this time, the map can be managed in layers and checked on the display unit H16 or can be returned to the initial map. [Tomioka Page 9, Paragraph 4]
Tomioka does not teach […] and performing a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information.
However, Chan does teach […] and performing a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information. (Chan Paragraph 0009: “In some embodiments, the step of updating the positioning information of the mobile vehicle in the new SLAM map further comprises: using a position and azimuth of the mobile vehicle at the current point to update the SLAM map to the new SLAM map and updating the positioning information, wherein the position and azimuth of the mobile vehicle are detected by the non-SLAM positioning device.”) (Note: Non-slam position = CAD information) (Chan Paragraph 0037: “In step S310, the mobile vehicle detects a first position trajectory and a first azimuth trajectory of the mobile vehicle on the SLAM map by a non-SLAM positioning device. Specifically, the non-SLAM positioning device can instantly record the position and azimuth of the mobile vehicle on the SLAM map.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel in view of Tomioka to include […] and performing a correction by replacing the position information of the associated group of feature points in the SLAM map with the position information of the visual index detected from the CAD information taught by Chan. This would have been for the benefit to provide a method for relocating a mobile vehicle in a SLAM map and a mobile vehicle to solve the problem of the mobile robot may not be able to relocate itself in the currently drawn SLAM map (i.e., the mobile robot gets lost). [Chan Paragraph 0004 and 0005]
8. Claim(s) 3 is/are rejected under 35 U.S.C. 103 as being unpatentable over Abel (DE 102021130631 A1) in view of Tomioka (WO 2019138834 A1) further in view of Chan (US 20210348941 A1) and further in view of Hesch (US 20150310310 A1).
Regarding claim 3, Abel in view of Tomioka further in view of Chan teaches claim 1, accordingly, the rejection of claim 1 is incorporated above.
Abel in view of Tomioka further in view of Chan does not teach The information processing device according to claim 1, wherein the detection unit generates an image associated with an arbitrary key frame of the map based on the CAD information and detects the visual index from the image.
However, Hesch does teach The information processing device according to claim 1, wherein the detection unit generates an image associated with an arbitrary key frame of the map based on the CAD information and detects the visual index from the image. (Hesch Paragraph 0005: “FIG. 2 is a diagram illustrating adjustment of a refined pose based on pose tracking of the electronic device of FIG. 1 in a free frame of reference in accordance with at least one embodiment of the present disclosure.”)
PNG
media_image2.png
459
379
media_image2.png
Greyscale
(Hesch Paragraph 0014: “FIGS. 1-9 illustrate various techniques for the determination of a pose of an electronic device within a local environment so as to support location-based functionality, such as augmented reality (AR) functionality, visual odometry or other simultaneous localization and mapping (SLAM) functionality, and the like. The term “pose” is used herein to refer to either or both of position (also referred to as a location) and orientation (also referred to as a point of view).”) (Hesch Paragraph 0016: “Concurrent with matching the generated set of descriptors, the electronic device can track changes in its pose (e.g. changes in its location, point of view, or both) in an arbitrary, or “free”, frame of reference (sometimes referred to herein as “free space”).”) (Hesch Paragraph 0026: “From this input data, the electronic device 100 can determine its relative pose without explicit absolute localization information from an external source. To illustrate, the electronic device 100 can perform analysis of the wide angle imaging camera image data 134 and the narrow angle imaging camera image data 136 to determine the distances between the electronic device 100 and the corners 124, 126, 128.”) (Hesch Paragraph 0045: “The datastores further can include a SLAM/AR datastore 442 that stores SLAM-based information, such as mapping information for areas of the local environment 112 (FIG. 1) already explored by the electronic device 100, or AR information, such as CAD-based representations of the relative locations of objects of interest in the local environment 112.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel in view of Tomioka further in view of Chan to include The information processing device according to claim 1, wherein the detection unit generates an image associated with an arbitrary key frame of the map based on the CAD information and detects the visual index from the image taught by Hesch. This would have been for the benefit to provide an enhanced technique of determining a relative position or relative orientation of an electronic device based on image-based identification of objects in a local environment of the electronic device. [Hesch Paragraph 0013]
9. Claim(s) 10-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Abel (DE 102021130631 A1) in view of Tomioka (WO 2019138834 A1) further in view of Chan (US 20210348941 A1) and further in view of Jing (CN 113124850 A).
Regarding claim 10, Abel in view of Tomioka further in view of Chan teaches claim 1, accordingly, the rejection of claim 1 is incorporated above.
Abel in view of Tomioka further in view of Chan does not teach The information processing device according to claim 1, wherein the correction unit corrects the map by replacing coordinates of the groups of feature points included in the constituent element for which the association has been made with coordinates of the visual index and fixing the coordinates and by optimizing coordinates of groups of feature points included in another constituent element using results of the fixing.
However, Jing does teach The information processing device according to claim 1, wherein the correction unit corrects the map by replacing coordinates of the groups of feature points included in the constituent element for which the association has been made with coordinates of the visual index and fixing the coordinates and by optimizing coordinates of groups of feature points included in another constituent element using results of the fixing. (Jing Page 11, Paragraph 6: “The following is based on how to correct the locating position information estimated by the laser sensor based on the locating code, the locating code provided by the embodiment of the invention comprises two kinds, one is the first type of locating code with position information, and the other one is the second type of locating code without position information; wherein the first type of positioning code is used for correcting the absolute coordinate of each positioning position information of the robot, so that the coordinate of the position point of each positioning position information of the robot in the target area is consistent with the coordinate of the position point in the coordinate system corresponding to the scene map;”) (Jing Page 16, Paragraph 5: “of course, if obtaining the second type of locating code, it has been obtained through the first type of locating code, the locating position information before obtaining the first type of locating code has been modified, the modified locating position information is changed into absolute position coordinate; then subsequently obtaining the second type of locating code with the same code, again determining the path error sum of the robot from the initial time to the current time, then based on the error sum and the error correction weight corresponding to each locating position information, correcting the position of each locating position information; so as to obtain the relative coordinate information of each locating position information in the coordinate system corresponding to the scene map.”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel in view of Tomioka further in view of Chan to include The information processing device according to claim 1, wherein the correction unit corrects the map by replacing coordinates of the groups of feature points included in the constituent element for which the association has been made with coordinates of the visual index and fixing the coordinates and by optimizing coordinates of groups of feature points included in another constituent element using results of the fixing taught by Jing. This would have been for the benefit to include a robot that includes a laser sensor, a vision sensor, and a control device, the control device comprises a position prediction module, a position correction module and a map generating module in order to solve the problem of a robot running path being quickly generated with high accuracy. [Jing Page 2, Paragraph 2 and Paragraph 4]
Regarding claim 11, Abel in view of Tomioka further in view of Chan teaches claim 1, accordingly, the rejection of claim 1 is incorporated above.
Abel in view of Tomioka further in view of Chan does not teach The information processing device according to claim 1, wherein the association unit generates a pair of sets of 3-dimensional coordinates by relating the visual index output by the detection unit to the feature point included in the SLAM map acquired by the map acquisition unit, and outputs the pair of sets of 3-dimensional coordinates as an association result to the correction unit.
However, Jing does teach The information processing device according to claim 1, wherein the association unit generates a pair of sets of 3-dimensional coordinates by relating the visual index output by the detection unit to the feature point included in the SLAM map acquired by the map acquisition unit, and outputs the pair of sets of 3-dimensional coordinates as an association result to the correction unit. (Jing Page 8, Paragraph 6: “position correction module 1032 is configured based on the visual sensor detects the set in the target area of the locating code, based on the locating code for locating position information of the robot predicted by the laser sensor for position correction, until obtaining the target area corresponding to the grid map.”) (Jing Page 8, Paragraph 7: “Here the positioning code can be used for the positioning position information of the robot to correct the identification code, which can be adhered on the ground, also can be adhered on the obstacle in the target area, such as adhered on the shelf, wall.”) (Jing Page 8, Paragraph 8: “Further, the positioning code here comprises a first type of positioning code for global correction of the positioning position information of the robot, the position coordinate of the target area of the first type of positioning code positioning is consistent with the coordinate of the scene map associated with the target area; so as to correct the locating position information of the robot based on the first type locating code, the coordinate information of each position point in the obtained grid map is the same as the coordinate information of the position point in the scene map, so that it is convenient for directly transferring the planned path information planned in advance to the grid map later.”) (Jing Page 11, Paragraph 6: “wherein the first type of positioning code is used for correcting the absolute coordinate of each positioning position information of the robot,”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel in view of Tomioka further in view of Chan to include The information processing device according to claim 1, wherein the association unit generates a pair of sets of 3-dimensional coordinates by relating the visual index output by the detection unit to the feature point included in the SLAM map acquired by the map acquisition unit, and outputs the pair of sets of 3-dimensional coordinates as an association result to the correction unit taught by Jing. This would have been for the benefit to include a robot that includes a laser sensor, a vision sensor, and a control device, the control device comprises a position prediction module, a position correction module and a map generating module in order to solve the problem of a robot running path being quickly generated with high accuracy. [Jing Page 2, Paragraph 2 and Paragraph 4]
Regarding claim 12, Abel in view of Tomioka further in view of Chan teaches claim 1, accordingly the rejection of claim 1 is incorporated above.
Abel in view of Tomioka further in view of Chan does not teach The information processing device according to claim 1, wherein the correction unit corrects 3-dimensional coordinates of the feature point included in the SLAM map acquired by the map acquisition unit based on 3-dimensional coordinates of the visual index related to the feature point on the SLAM map by the association unit.
However, Jing does teach The information processing device according to claim 1, wherein the correction unit corrects 3-dimensional coordinates of the feature point included in the SLAM map acquired by the map acquisition unit based on 3-dimensional coordinates of the visual index related to the feature point on the SLAM map by the association unit. (Jing Page 5, Paragraph 4:“obtaining the absolute coordinate information of each of the positioning position information in the coordinate system corresponding to the scene map.”) (Jing Page 7, Paragraph 3: “if based on the synchronous locating and mapping (SLAM) mode for locating the generated grid map,”) (Jing Page 7, Paragraph 4: “when obtaining the position information of the robot based on SLAM mode, it is easy to generate accumulated error, so the position correction module can be based on the positioning code set in the target area obtained by the vision sensor to perform position correction for each positioning position information estimated by the laser sensor; so as to obtain the accurate grid map corresponding to the target area, then the planning route information planned in the scene map corresponding to the target area can be transferred to the grid map, for example, planning based on the set area corresponding to the CAD map in advance;”) (Jing Page 11, Paragraph 5: “Further, after obtaining the positioning position information of the robot at the current time, it can be based on the positioning position information of the robot at the current time and the relative position information of different target obstacle at the current time and the robot, determining the position information of the different target obstacle at the current time;”)
Therefore, it would have been obvious to one of ordinary skill in art before the effective filing date of the claimed invention to have modified Abel in view of Tomioka further in view of Chan to include The information processing device according to claim 1, wherein the correction unit corrects 3-dimensional coordinates of the feature point included in the SLAM map acquired by the map acquisition unit based on 3-dimensional coordinates of the visual index related to the feature point on the SLAM map by the association unit taught by Jing. This would have been for the benefit to include a robot that includes a laser sensor, a vision sensor, and a control device, the control device comprises a position prediction module, a position correction module and a map generating module in order to solve the problem of a robot running path being quickly generated with high accuracy. [Jing Page 2, Paragraph 2 and Paragraph 4]
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN J HARVEY whose telephone number is 571-272-5327. The examiner can normally be reached 8:00AM-5:00PM M-Th, 8:00AM-4:00PM F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kito Robinson can be reached at 571-270-3921. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/K.J.H./Junior Patent Examiner, Art Unit 3664
/KITO R ROBINSON/Supervisory Patent Examiner, Art Unit 3664
Prosecution Insights