DETAILED ACTION
Claims 1-3 received on 07/31/2025 are considered in this office action. Claims 1-3 are pending for examination.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 07/31/2025 is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3 are rejected under 35 U.S.C. 103 as being unpatentable over Cesic (US20220121207A1), in view of Yerli (US 20230347923 A1).
Regarding claim 1, Cesic teaches a system for creating a virtualized meta-map using spatial data acquired by an intelligent indoor autonomous mobile robot (FIG. 1; FIG. 6; para. [0026]: “Environment 100 includes […] autonomous mobile robot 140”; para. [0041]: “environment map database 240 may be updated by central communication facility 130 based on information received from operator device 110 and/or from the robots 140. Information may include images, depth information, auxiliary information, semantic information, instance information, and any other information described herein”; para. [0082]: “The map within user interface 600 may be updated as feedback is received from one or more robots by central communication system 130 in any manner described herein. The map may be any form of map described herein (e.g., semantic map, instance map, etc.). The map may be two-dimensional or three-dimensional, and where the map is three-dimensional”, wherein map indicates virtualized meta-map), the system comprising:
an autonomous mobile robot or mobility configured to transmit position information of a target object in an indoor space acquired using a visualization sensor during an autonomous travel process (FIGs. 6-7; para. [0040]: ‘Central communication facility 130 may receive a map from operator device 110, or may generate one based on input received from one or more robots 140 (e.g., by stitching together images and/or depth information received from the robots as they traverse the facility,”; para. [0042]: “The semantic information may identify classes of objects within the maps. For example, the map may show, that for a given object, the object is of a given class, such as “pallet”, “obstacle,” “human,” “robot,” “pallet stand,” and so on. The instance information may indicate the boundaries of each object. […] central communication facility 130 (in processing raw image data transmitted from the robot to the central communication facility) recognizes an object in space”; para. [0046]: “identifies an object in proximity to the robot. The sensors may include one or more cameras, one or more depth sensors, one or more scan sensors (e.g., RFID), a location sensor (e.g., showing location of the robot within the facility and/or GPS coordinates)”, wherein map is updated based on the sensor data obtained from robots, which obtains object information and location as shown in FIGs. 6-7); and
a cloud server (FIG. 1 Central Communication System; para. [0029]: “Central communication system 130 acts as a central controller for a fleet of one or more robots including autonomous mobile robot 140 […] Central communication system 130 may be located within environment 100 or may be located outside of environment 100 (e.g., in a cloud implementation)”, wherein “central controller” indicates server) configured to:
generate, in advance, a virtualized map including object information of the autonomous mobile robot to which overall landscape and object information of the indoor space are mapped (FIG. 6-7; para. [0082]: “The map within user interface 600 may be updated as feedback is received from one or more robots by central communication system 130 in any manner described herein”, wherein FIG. 6-7 shows overall landscape and object information of the indoor space are mapped);
display the autonomous mobile robot or mobility and position information of the target object in a virtual space (FIG. 6-7; para. [0082]: “User interface 600 may be generated for display to an operator by way of operator device 110. User interface 600 may be generated by central communication system 130, an application installed on operator device 110, or a combination thereof […] icon 630 corresponds to a robot,”; para. [0084]: “The user may select source icon 720. Source icon 720 may be a single object or may include multiple objects.”);
generate a meta-map reflecting a travel instruction for the autonomous mobile robot or mobility to move to the target object in the virtual space (para. [0085]: “establish a mission where a robot is to transport one or more objects from a source to a destination, the user may perform a drag-and-drop motion from source icon 720 to destination icon 730. Such drag-and-drop motion is detected (e.g., by the application and/or central communication system 130), and responsively, the mission is defined to have the selected robot transport the object from the source to the destination”);
provide information (FIG. 2; FIG. 14; para. [0098]: “communication provides 1408 an instruction to each selected robot to transport the plurality of objects from the source area to the destination area (e.g., using robot instruction module 234)”; para. [0039]: “In an embodiment, robot instruction module 234 includes detailed step-by-step instructions on how to transport the objects. In another embodiment, robot instruction module 234 transmits a general instruction to transmit one or more objects from the source area to the destination area”),
wherein the autonomous mobile robot or mobility moves to the target object in the indoor space on the basis of the information (FIGs. 6-7; para. [0085]: “The selected robot thereafter carries out the mission consistent with the manners described above”), but fails to specifically teach convert a virtual space coordinate system in the meta-map into a real-world coordinate system and information obtained by converting the virtual space coordinate system in the indoor space into the real-world coordinate system.
However, in the same field of endeavor, Yerli teaches convert a virtual space coordinate system in the meta-map into a real-world coordinate system and provide information obtained by converting the virtual space coordinate system in the indoor space into the real-world coordinate system to the autonomous mobile robot or mobility (FIG. 2; FIG. 5; Abstract: “The instructions cause the at least one server computing device to provide a navigation engine configured to compute a navigation route of at least one autonomous mobile robot using at least the virtual three-dimensional coordinates of the at least one virtual object, enabling the at least one autonomous mobile robot to autonomously navigate the physical environment by reference to a virtual replica of the physical environment comprised in the virtual world system”; para. [0057]: “least one autonomous mobile robot to autonomously navigate the real world through a virtual replica of the at least one autonomous mobile robot comprised in the virtual world system”; para. [0043]: “example illustration of FIG. 2, the robot 212 climbs the stairs 220 by receiving and executing data and instructions processed by at least one server computing device hosting the virtual world system and navigation engine comprising a navigation route”, wherein using virtual three-dimensional coordinates to autonomously navigate the physical environment comprises a step of convert a virtual space coordinate system in the meta-map into a real-world coordinate system); and
wherein the autonomous mobile robot or mobility moves to the target object in the indoor space on the basis of the information (FIG. 2; FIG. 5; para. [0043]: “FIG. 2, the robot 212 climbs the stairs 220 by receiving and executing data and instructions processed by at least one server computing device hosting the virtual world system and navigation engine comprising a navigation route”; Abstract: “autonomous mobile robot to autonomously navigate the physical environment by reference to a virtual replica of the physical environment comprised in the virtual world system”).
Cesic and Yerli are considered to be analogous to the claimed invention because both are in the same field of controlling autonomous robots by generating virtual map/model of an environment. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the central communication system of Cesic and incorporate the teachings of Yerli and generate navigational instructions based on virtual third dimension coordinates. Doing so will assist in a smoother navigation by the at least one autonomous mobile robot, decreasing chances of collision or damages to structures of the internal or external environment, people or the autonomous mobile robot itself by enabling location-based autonomous navigation using a virtual world system (Yerli, para. [0024])
Regarding claim 2, Cesic in view of Yerli teaches the system of claim 1. The combination of Cesic in view of Yerli further teaches wherein the cloud server (FIG. 1 Central Communication System; Yerli FIG. 1 102; Yerli para. [0026]: “one server computing device 102”) comprises:
an information collection unit configured to collect position information and feature information of a person and/or an object measured during the autonomous travel process of the autonomous mobile robot or mobility and real-world coordinate information of the indoor space in which the autonomous mobile robot or mobility travels from an external server (Cesic FIG. 6-7, 11A-12B; Cesic para. [0040]: ‘Central communication facility 130 may receive a map […] generate one based on input received from one or more robots 140 (e.g., by stitching together images and/or depth information received from the robots as they traverse the facility,”; Cesic para. [0042]: “The semantic information may identify classes of objects within the maps. For example, the map may show, that for a given object, the object is of a given class, such as “pallet”, “obstacle,” “human,” “robot,” “pallet stand,” and so on. The instance information may indicate the boundaries of each object. […] central communication facility 130 (in processing raw image data transmitted from the robot to the central communication facility) recognizes an object in space”; Cesic para. [0046]: “identifies an object in proximity to the robot. The sensors may include one or more cameras, one or more depth sensors, one or more scan sensors (e.g., RFID), a location sensor (e.g., showing location of the robot within the facility and/or GPS coordinates)”, wherein map is updated based on the sensor data obtained from robots, which obtains object information and location as shown in FIGs. 6-7 and these information is obtained via network, thus indicating external server);
a virtual space modeling unit configured to model a three-dimensional (3D) virtual space on indoor coordinates of the indoor space using a digital twin technique (Cesic FIG. 13; Cesic para. [0082]: “The map may be two-dimensional or three-dimensional, and where the map is three-dimensional”; Yerli FIG. 2; Yerli para. [0033]: “the virtual replicas of the current disclosure refer to a virtual version, or virtual twin, of a real-world element, or real twin, which may either be mapped or modeled through computer assisted drawing (CAD) or computer-assisted engineering (CAE) methods, input by computer models of the real world objects 116 (e.g., building information models—BIMs) or other suitable methods, and which may mirror not just the appearance but also the behavior of the real twin”);
a travel information agent unit configured to provide the travel instruction for an avatar of the autonomous mobile robot or mobility which is an agent in the virtual space to move to the target object (Cesic FIG. 6-7; Cesic para. [0039]: “Robot instruction module 234 transmits instructions to the selected one or more robots”; Cesic para. [0082]: “User interface 600 may be generated for display to an operator by way of operator device 110. User interface 600 may be generated by central communication system 130, an application installed on operator device 110, or a combination thereof […] icon 630 corresponds to a robot,”; Cesic para. [0085]: “establish a mission where a robot is to transport one or more objects from a source to a destination, the user may perform a drag-and-drop motion from source icon 720 to destination icon 730.”; Yerli Fig. 2; Yerli para. [0057]: “least one autonomous mobile robot to autonomously navigate the real world through a virtual replica of the at least one autonomous mobile robot comprised in the virtual world system”, wherein virtual replica indicates an avatar of the autonomous mobile robot or mobility); and
a meta-map generation unit configured to generate a meta-map including the travel instruction for the avatar of the autonomous mobile robot or mobility to move to the target object in the virtual space (Cesic FIG. 6-7; Yerli Fig. 2; Yerli FIG. 5; Yerli Abstract: “compute a navigation route of at least one autonomous mobile robot using at least the virtual three-dimensional coordinates of the at least one virtual object”; Yerli para. [0057]: “least one autonomous mobile robot to autonomously navigate the real world through a virtual replica of the at least one autonomous mobile robot comprised in the virtual world system”, wherein Cesic teaches providing instructions to the mobile robot based on the map and Yerli’s virtual replica performs navigation in the virtual world system, thus indicating generate a meta-map including the travel instruction).
Regarding claim 3, Cesic in view of Yerli teaches the system of claim 2. The combination of Cesic in view of Yerli further teaches wherein the meta-map generation unit comprises:
a mapping unit configured to generate a mapping map in which spatial coordinates of the virtual space which is a digital twin space (DTS) generated by the virtual space modeling unit are mapped to spatial coordinates of a real world (Cesic FIG. 6-7; Cesic para. [0082]: “The map within user interface 600 may be updated as feedback is received from one or more robots by central communication system 130 in any manner described herein); Yerli FIG.2; Yerli para. [0009]: “the at least one virtual object is spatially defined by virtual three-dimensional coordinates that correspond to three-dimensional coordinates of a corresponding physical object in a physical environment”; Yerli para. [0033]: “virtual twin, of a real-world element, or real twin, which may either be mapped or modeled [ […] synchronizing the virtual replicas with their respective real counterpart”);
a synchronization unit configured to synchronize position information of the autonomous mobile robot or mobility in the real world on the mapping map with a position of an avatar of the autonomous mobile robot or mobility in the virtual space (Cesic FIG. 6-7; Yerli FIG. 2; Yerli para. [0033]: “virtual twin, of a real-world element, or real twin, which may either be mapped or modeled […] synchronizing the virtual replicas with their respective real counterpart”, wherein the position of the robot is shown in the model/map);
a target object designation unit configured to designate or generate the target object at a position designated by an external terminal in the mapping map (Cesic FIG. 6-7; Cesic para. [0083]-[0085]: “User interface 700 shows robot icon 710, source icon 720, and destination icon 730, each of which are selected by a user […]establish a mission where a robot is to transport one or more objects from a source to a destination, the user may perform a drag-and-drop motion from source icon 720 to destination icon 730. Such drag-and-drop motion is detected (e.g., by the application and/or central communication system 130), and responsively, the mission is defined to have the selected robot transport the object from the source to the destination”; Cesic para. [0027]: “Operator device 110 may be any client device that interfaces one or more human operators with one or more autonomous mobile robots of environment 100 and/or central communication system 13”; Yerli para. [0034]: “the navigation destination data 126 is input through a user device 134, such as a mobile devices”);
a movement route generation unit configured to calculate a movement route from the position of the autonomous mobile robot or mobility to the target object in the mapping map and provide the calculated movement route (Cesic para. [0039]: “robot instruction module 234 includes detailed step-by-step instructions on how to transport the objects. In another embodiment, robot instruction module 234 transmits a general instruction to transmit one or more objects from the source area to the destination area, leaving the manner in which the objects will be manipulated and ultimately transmitted up to the robot to determine autonomously”; Yerli para. [0012]: “receiving a navigation route computed by the navigation engine of the at least one server computing device using at least virtual three-dimensional coordinates of virtual objects in the virtual world system that correspond to three-dimensional coordinates of corresponding physical objects in a physical environment and navigation destination data; and autonomously navigating the physical environment using the navigation route”, wherein navigation route is first generated in virtual world system); and
a meta-map provision and update unit configured to generate a meta-map reflecting the movement route generated by the movement route generation unit and reflecting the travel instruction provided by the travel information agent unit in the movement route and transmit the meta-map to the autonomous mobile robot or mobility in the real world (Cesic 6-7; Cesic FIG. 9A-9D; Cesic para. [0039]: “robot instruction module 234 includes detailed step-by-step instructions on how to transport the object”; Yerli FIG. 5; Yerli para. [0012]: “receiving a navigation route computed by the navigation engine of the at least one server computing device using at least virtual three-dimensional coordinates of virtual objects in the virtual world system that correspond to three-dimensional coordinates of corresponding physical objects in a physical environment and navigation destination data; and autonomously navigating the physical environment using the navigation route”, wherein Cesic generates a trajectory based on the “map” and displays the robot’s movement in the UI, and Yerli teaches performing navigation route in the virtual world system, thus indicating configured to generate a meta-map reflecting the movement route and transmitting the meta-map),
wherein the meta-map provision and update unit updates the existing meta-map with a meta-map in which a movement route to the target object is recalculated to reflect detection information, which is obtained by detecting an obstacle (moving object) while the autonomous mobile robot or mobility moves along the movement route displayed on the meta-map, in real time and provide the updated meta-map (Cesic FIGs. 6-7 ,9A-9D, 11A-11B; ; Cesic para. [0060]: “The route may be selected and updated by the robot using navigation module 335, based on information derived from environment 240 and environment information determined by way of capturing and processing images along the route (e.g., encountering unexpected obstacles such as objects, human beings, etc.)”; Cesic para. [0043]: “Central communication facility 130 may continuously update the maps as such information is received”; Cesic para. [0068]: “The alert may also include a next step that the robot will take”; Yerli FIG. 2; Yerli FIG. 5, wherein map and route will be updated as objects are detected during traveling of robot, thus indicating update unit updates the existing meta-map).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
FONG (US20190212752A1) teaches controlling the mobile cleaning robot to navigate in the environment using the persistent map and sensing data provided by the at least one sensor, sharing the persistent map with a second mobile cleaning robot, and coordinate with the second mobile cleaning robot to perform cleaning tasks.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ANDREW S KIM whose telephone number is (571)272-7356. The examiner can normally be reached Mon - Fri 8AM - 5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, James J Lee can be reached on (571) 270-5965. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANDREW SANG KIM/Examiner, Art Unit 3668
Prosecution Insights