Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This Office action is in response to the amendment filed on 01/30/2026. Claims 1-2, 5-8, 10-12, 14-17, and 19-20 are currently pending with claims 1, 11, and 16 being amended, and claims 3-4, 9, 13, and 18 being cancelled.
Response to Amendment
The amendments to the claims submitted on 01/30/2026 overcome the claim objections set forth in the previous Office action except for those set forth in the claim objection section.
Response to Arguments
Applicant's arguments filed 01/30/2026 have been fully considered but they are not persuasive. The applicant asserts that the prior art does not anticipate the limitation of “causing the first one of the subset of robotic edge devices to move outside the edge communication range to perform the first one of the physical activities without being connected to a network and without edge computation or edge communication,”. The examiner disagrees with this assertion for at least the following reasons. Regarding the performance of a “physical activity” the broadest reasonable interpretation of this may include operation of the robot to move between locations (i.e. turning wheels, operating sensors to scan an environment, etc.) as well as an operation at a target location such as disclosed by Brazeau in at least paragraph 0023. Furthermore, the examiner would note that the currently provided claim language does not require all activities to be performed without connection to the network. So the system performing any physical task while not in active communication would anticipate this limitation. Regarding the applicants assertion that the prior art does not anticipate performing a task while not in communication the examiner directs attention to Brazeau’s figure 1 (shown below with annotations). This clearly shows mobile unit 104 moving between a plurality of locations, some of which include a local network which overlaps with the network of the mobile unit allowing for communication between the mobile unit and the management module (item 102) or the workstation computing device (item 112) via the communication mechanism (item 106) at locations which have a communication mechanism. The mobile unit also approaches the inventory holder (item 108) which includes an inventory identifier (item 110). This location does not have a communication mechanism present which the mobile unit may connect to while performing the activities discussed in at least paragraph 0023 (identification of the inventory holder, identification of items present in the inventory holder, retrieval of the inventory holder, etc.).
[AltContent: textbox (Location C:
mobile drive unit 104
communication mechanism 106
workstation computing device 112)][AltContent: textbox (Location B:
mobile drive unit 104
inventory holder 108
inventory holder identifier 110)][AltContent: textbox (Location A:
mobile drive unit 104
communication mechanism 106
management module 102)]
PNG
media_image1.png
686
500
media_image1.png
Greyscale
For clarity of record and convenience, the examiner has included paragraphs 0021-0026 of Brazeau below with emphasis added. This disclosure indicates that the mobile unit connects to a first network at location A, travels to location B to perform tasks, then travels to location C and connects to a second network.
“[0021] FIG. 1 illustrates an example inventory system in which task assignments may be provided to mobile drive units via ad hoc networks in accordance with some embodiments. In FIG. 1, a computing device including a management module 102 may be in communication with a mobile drive unit 104. The computing device including the management module 102 may be configured to communicate one or more sets of computer executable instructions (e.g., task assignments) to the mobile drive unit 104 via a communication mechanism 106. Examples of a communication mechanism may include one or more radio frequency (RF) transceivers configured to send and receive communications using near-field communications (NFC), or other radio frequency or wireless communication protocols such as Bluetooth, Bluetooth low-energy (BLE), a wireless local area network (e.g., WiFi), iBeacon, etc. In some embodiments, communication mechanism 106 may include an infrared communication device. In some embodiments, the communication mechanism 106 may include both long range and short range communication means. For example, the communication mechanism may include an antenna configured to connect to a cellular network in order to enable communication with various other components of the depicted inventory system. In addition, the inventory system may include a number of inventory holders 108, each of which may be identified via an inventory holder identifier 110. The inventory system may additionally include a workstation computing device 112.
[0022] In some embodiments, the computing device executing the management module 102 may discover the presence of a mobile drive unit 104. For example, the mobile drive unit 104 may connect to a private network when it is within range of the short range communication mechanism 106. Upon detecting that the mobile drive unit 104 has connected to the private network, the computing device including the management module 102 may be configured to receive a report on the status of the mobile drive unit 104, identify a current status of the mobile drive unit, identify an appropriate task assignment for the mobile drive unit, generate instructions to result in the completion of the task assignment by the mobile drive unit 104, and transmit the task assignment to the mobile drive unit 104. In some embodiments, the management module 102 may determine what task assignment is appropriate for a mobile drive unit based on the type and/or capabilities associated with the mobile drive unit. For example, the management module 102 may receive an identifier associated with the mobile drive unit and may determine, based on that identifier, a type and/or identity of the mobile drive unit. The management module 102 may then query a database of mobile drive units to identify the detected mobile drive unit and its capabilities.
[0023] In some embodiments, a task assignment may include an identification of an inventory holder 108, an identification of one or more items in the inventory holder 108 to be retrieved, and an identification of a workstation. The task assignment may include instructions that cause a mobile drive unit 104 to retrieve the identified inventory holder, bring the retrieved inventory holder to the identified workstation, and provide the indication of the one or more items to the workstation to be conveyed to an administrator. Upon receiving a task assignment from the computing device executing the management module 102, the mobile drive unit 104 may generate a route to an indicated inventory holder 108 and subsequently to an indicated workstation.
[0024] Upon arriving at the inventory holder 108, the mobile drive unit may verify that the inventory holder 108 is the indicated inventory holder based on an inventory holder identifier located on or near that inventory holder 108. The mobile drive unit 104 may then retrieve the inventory holder 108 or one or more items located within the inventory holder 108 in accordance with the task assignment. The mobile drive unit 104 may then move to the indicated workstation with the retrieved inventory holder 108.
[0025] Upon arriving at the indicated workstation, the mobile drive unit 104 may connect to a second network operated by a workstation computing device 112. Upon connecting to the second network, the mobile drive unit 104 may communicate one or more instructions to the a workstation computing device 112 via the second private network. For example, the mobile drive unit 104 may communicate an indication of the one or more items to be removed from the inventory holder 108 to the workstation computing device 112. In some embodiments, an administrator or other operator may remove the items from the inventory holder and provide a status update to the mobile drive unit 104. For example, if the operator is unable to locate an indicated item, the operator may update a status with the mobile drive unit 104, via the second network, to indicate that the item was not found. The mobile drive unit 104 may then generate a status alert to be provided to the computing device executing the management module 102.
[0026] In accordance with at least some embodiments, the mobile drive units may each include a communication mechanism. In these embodiments, the mobile drive unit may operate a network. Various system components may each have a wireless transmitter/receiver and may connect to a network operated by the mobile drive unit 104 as the mobile drive unit comes within transmission range of the system components. In some embodiments, each system component may announce its identifier over the network operated by the mobile drive unit as it connects to the network, which the mobile drive unit 104 may use to determine if the system component is relevant. In some embodiments, the mobile drive unit 104 may only allow certain system components to connect to its network.”
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-2, 5-8, 10-12, 14-17, and 19-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Griffin et al. (US 20210232136 A1), hereinafter Griffin in view of Banjade et al. (US 20220014923 A1), hereinafter Banjade, and Brazeau et al. (US 20180113475 A1), hereinafter Brazeau.
Regarding claim 1, Griffin teaches:
1. (Currently Amended) A computer-implemented method comprising:
identifying a plurality of robotic edge devices in a geographic location; (Paragraph 0032,"As used herein, a robot network or network of robots may comprise a plurality of robots communicatively coupled to each other and/or coupled to an external cloud server. The plurality of robots may communicate data to other robots on the robot network and/or to an external cloud server. The plurality of robots on the robot network may include robots of different or the same functionalities. A robot network communicating with a server may comprise a plurality of robots on the robot network communicating with the server." and Paragraph 0056, "This odometry may include robot 102's position (e.g., where position may include robot's location, displacement and/or orientation, and may sometimes be interchangeable with the term pose as used herein) relative to the initial location. Such data may be stored in data structures, such as matrices, arrays, queues, lists, arrays, stacks, bags, etc. According to exemplary embodiments, the data structure of the sensor data may be called an image.")
determining attributes for each robotic edge device of the plurality of robotic edge devices (Paragraph 0121, "FIG. 9 illustrates a data table 900 comprising data on a plurality of robots 102 on a robot network 304 and properties thereof, according to an exemplary embodiment. Each robot 102 may be assigned a robot ID distinguishing each of the plurality of robots 102. Each robot 102 may comprise a plurality of properties including, but not limited to, properties of a processing device of each robot 102 (e.g., clock rate, number of cores, etc.), a maximum speed of the robots 102 (e.g., meters per second, feet per second, etc.), a number of cameras on the robots 102, a functionality of the robots 102, and/or any other additional properties of the robots 102. Data table 900 may comprise N robots, wherein index N may be any non-zero integer number corresponding the number of robots 102 on the robot network 304. Similarly, index I may correspond to an arbitrary integer number of cameras on the N.sup.th robot 102 greater than or equal to zero. Data table 900 may be stored in a memory 132 of a cloud server 202 and may be accessed by a processing device 130 of the cloud server 202 to determine and distribute instructions to the plurality of robots 102 on the robot network 304."), … ;
identifying a task to be performed at the geographic location by the plurality of robotic edge devices, the task including a plurality of physical activities; (Paragraph 0041, "According to at least one non-limiting exemplary embodiment, a system comprising a cloud server communicatively coupled to a robot network comprising a plurality of robots is disclosed. The cloud server may be configurable to receive an input from an operator and generate an output to the operator based on data gathered by the plurality of robots on the robot network. The cloud server may be further configurable to generate an instruction to be executed by the robot network, the instruction may configure the robot network to compute and/or collect data necessary to respond to the operator input." and Paragraph 0069, " For example, an external device 206 may be a user interface, wherein data from the user interface may comprise a request for data, a physical task to be performed by the robots, and/or a request for a computation to be performed by the plurality of robots 102. The assigned individualized tasks to the robots 102 coupled to the cloud server 202 may enhance the efficiency of the cloud server 202 to respond to an input from the external device 206, as illustrated below in FIG. 6-8, as the work load required to respond to the input may be distributed among the plurality of robots 102." Examiner Note: The limitation of “physical activities” under BRI may encompass activities such as travelling to a secondary location, moving components in order to survey the environment, grasping/placing objects etc.)
determining, based on the attributes, a subset of robotic edge devices that are capable of completing the task; and
assigning the subset of robotic edge devices to complete the task (Paragraph 0011, "Example embodiments disclosed herein are directed to methods, systems and non-transitory computer readable mediums that may have computer readable instructions stored thereon, that when executed by at least one processor or processing device, configure the at least one processing device to, receive an operator input comprising instructions for a robot network, the robot network comprising a plurality of independently operable robots that are communicatively coupled to each other in an environment; transmit the instructions to at least a first sub-set of robots in the robot network such that the first sub-set of robots execute the instructions in the environment, the instructions comprising a plurality of tasks to be performed by the first sub-set of robots such that a respective task of the plurality of tasks is assigned to a respective robot of the first sub-set of robots based on bandwidth of the respective robot; receive data collected by the first sub-set of robots during simultaneous performance of the plurality of tasks by the first sub-set of robots in the environment; and generate an operator output based on the data collected by the first sub-set of robots.");
initiating the subset of robotic devices to complete the task; … (Paragraph 0012, " The methods, systems and non-transitory computer readable mediums disclosed herein are further configurable to execute the computer readable instructions to, transmit the instructions to the first sub-set of robots in the robot network only if response to the operator input is not previously stored in the memory. Wherein, the transmission of the instructions to the first sub-set of robots configures the respective robot to navigate from a first location to a second location and collect data on one or more items at the second location, and transmit the data collected on the one or more items to the at least one controller. Further, wherein the transmission of the instructions to the first sub-set of robots configures the respective robot to, retrieve one or more items from a designated pick-up location and drop the one or more items at a designated drop-off location, and transmit data to the at least one processing device simultaneously as the one or more items are relocated from the pick-up location to the drop-off location.")
determining that a first one of the physical activities is located (Paragraph 0012, “The methods, systems and non-transitory computer readable mediums disclosed herein are further configurable to execute the computer readable instructions to, transmit the instructions to the first sub-set of robots in the robot network only if response to the operator input is not previously stored in the memory. Wherein, the transmission of the instructions to the first sub-set of robots configures the respective robot to navigate from a first location to a second location and collect data on one or more items at the second location, and transmit the data collected on the one or more items to the at least one controller. Further, wherein the transmission of the instructions to the first sub-set of robots configures the respective robot to, retrieve one or more items from a designated pick-up location and drop the one or more items at a designated drop-off location, and transmit data to the at least one processing device simultaneously as the one or more items are relocated from the pick-up location to the drop-off location.”) …
determining that a first one of the subset of robotic edge devices can perform the first one of the physical activities (Paragraph 0130, “According to at least one non-limiting exemplary embodiment, wherein data table 1000 is stored in memory 132 of a cloud server 202, the cloud server 202 may access the data table 1000 to distribute tasks to robots 102 in a robot network 304 best suited to perform the task. Similarly, the cloud server 202 may access the data table 1000 to distribute tasks to the robots 102 on the robot network to satisfy a plurality of instructions simultaneously. For example, instruction 1 and its comprising tasks may be assigned to a first set of robots 102, instruction 2 and its comprising tasks may be assigned to a second set of different robots 102, and so forth. Additionally, the cloud server 202 may utilize data from completed tasks during execution of some instructions to satisfy, at least in part, other instructions and their comprising tasks.”) …
Griffin does not specifically teach the device attributes being computation latency or communication range or assigning a task to a device which leaves the network to complete the task before reconnecting. However, Banjade, in the same field of endeavor of robotic edge devices, teaches:
… the attributes including edge computation latency and edge communication range (Paragraph 0051, "In an embodiment, the selecting module 223 may be configured for selecting at least one edge device from a plurality of edge devices within a communication range of the IoT device based on network conditions and computational latency associated with the plurality of edge devices. In an embodiment, the identifying module 225 may be configured for identifying the preferred network for connecting the IoT device with the selected edge device based on available bandwidth and historical inference time records of a plurality of networks associated with the IoT device." also see Paragraph 0115, "Communications from any IoT device 902 may be passed along a convenient path between any of the IoT devices 902 to reach the gateways 904. In these networks, the number of interconnections provide substantial redundancy, allowing communications to be maintained, even with the loss of a number of IoT devices 902. Further, the use of a mesh network may allow IoT devices 902 that are very low power or located at a distance from infrastructure to be used, as the range to connect to another IoT device 902 may be much less than the range to connect to the gateways 904." which discusses that some devices communicate through the mesh due to having a smaller communication range. This demonstrates that the system is aware of the range of each device and may select a device based on this characteristic. Please also see paragraphs 0133 and 0026.) …
However, Brazeau, in the same field of endeavor of robotics, teaches:
… outside the edge communication range; (Paragraph 0016, “Embodiments herein are directed to an inventory system that includes mobile drive units and/or other robotic components managed via a series of local networks. Specifically, features herein are directed to a series of separate wireless local networks, each operated by various system components. Upon entering within transmission range of a wireless local network, a mobile drive unit may connect to the local network, identify the system component associated with the local network, determine if the mobile drive unit has been provided with any instructions directed to the system component, and execute one or more instructions related to the identified system component. A system component may be any resource or device that performs a function on behalf of the inventory system. By way of non-limiting example, a system component may comprise a robotic device, an input sensor, an inventory holder, a workstation, or any other suitable system resource.”) …
without requiring edge computation in parallel with performance of the first one of the physical activities; (Paragraph 0081, “FIG. 8 illustrates an example interaction between a computing device executing a management module, a mobile drive unit, and a computing device associated with a system component in accordance with at least some embodiments. In FIG. 8, a computing device executing a management module 802 may operate a first local network 804. As a mobile drive unit 806 enters the vicinity of the first local network (e.g., comes within transmission range of a wireless transmitter/receiver), the mobile drive unit 806 is discovered by the computing device 802. Upon discovery, the computing device 802 identifies the next task in a queue of tasks to be completed which the mobile drive unit 806 is capable of completing. In some embodiments, the computing device 802 generates instructions 808 (e.g., a task assignment) to be provided to each system component involved in the completion of the task. For example, the computing device 802 may generate instructions to be executed by the mobile drive unit 806 that include an initial route, inventory holder identifier, and workstation identifier. The computing device 802 may also generate instructions to be executed by a loader device that indicates the inventory holder identifier to be loaded onto the mobile drive unit 806. Additionally, the computing device 802 may generate instructions to be executed by a workstation computing device. In some embodiments, each of the generated instructions may be provided to the mobile drive unit 806, and may subsequently be distributed to each computing device 810 associated with each respective system component by the mobile drive unit 806 as it comes into proximity of that system component.” Please also see figures 1 and 8. This demonstrates that instructions are communicated but the robot is not in constant connection to the network at all points during the assigned task. Physical activities such as control of the system to move between locations and performance of retrieval tasks are performed while outside of the communication range of other system components.) …
causing the first one of the subset of robotic edge devices to move outside the edge communication range to perform the first one of the physical activities without being connected to a network and without edge computation or edge communication; and (Paragraphs 0017-0018, “In accordance with an embodiment, a mobile drive unit may approach a central authority to receive instructions. The central authority may identify a task that the mobile drive unit is capable of performing and may generate a task assignment based on the identified task.
The task assignment may comprise a set of instructions that cause the mobile drive unit to perform various actions with respect to various system components (e.g., robotic devices, input sensors, inventory holders, etc.). In some embodiments, one or more wireless local networks may be operated with respect to these various system components. The mobile drive unit may be configured (via the task assignment) to visit each of these system components in turn and execute the actions indicated in the task assignment. As the mobile drive unit enters within a proximity of the system component, it may connect to the wireless local network associated with that system component. Upon establishing a connection to the wireless local network, the mobile drive unit may transmit instructions to cause the system component to complete the action indicated in the task assignment with respect to that system component.” As well as Paragraphs 0023-0024, “In some embodiments, a task assignment may include an identification of an inventory holder 108, an identification of one or more items in the inventory holder 108 to be retrieved, and an identification of a workstation. The task assignment may include instructions that cause a mobile drive unit 104 to retrieve the identified inventory holder, bring the retrieved inventory holder to the identified workstation, and provide the indication of the one or more items to the workstation to be conveyed to an administrator. Upon receiving a task assignment from the computing device executing the management module 102, the mobile drive unit 104 may generate a route to an indicated inventory holder 108 and subsequently to an indicated workstation.
Upon arriving at the inventory holder 108, the mobile drive unit may verify that the inventory holder 108 is the indicated inventory holder based on an inventory holder identifier located on or near that inventory holder 108. The mobile drive unit 104 may then retrieve the inventory holder 108 or one or more items located within the inventory holder 108 in accordance with the task assignment. The mobile drive unit 104 may then move to the indicated workstation with the retrieved inventory holder 108.”)
causing the first one of the subset of robotic edge devices to move back inside the edge communication range upon determining completion of the first one of the physical activities. (Paragraph 0022, “In some embodiments, the computing device executing the management module 102 may discover the presence of a mobile drive unit 104. For example, the mobile drive unit 104 may connect to a private network when it is within range of the short range communication mechanism 106. Upon detecting that the mobile drive unit 104 has connected to the private network, the computing device including the management module 102 may be configured to receive a report on the status of the mobile drive unit 104, identify a current status of the mobile drive unit, identify an appropriate task assignment for the mobile drive unit, generate instructions to result in the completion of the task assignment by the mobile drive unit 104, and transmit the task assignment to the mobile drive unit 104. In some embodiments, the management module 102 may determine what task assignment is appropriate for a mobile drive unit based on the type and/or capabilities associated with the mobile drive unit. For example, the management module 102 may receive an identifier associated with the mobile drive unit and may determine, based on that identifier, a type and/or identity of the mobile drive unit. The management module 102 may then query a database of mobile drive units to identify the detected mobile drive unit and its capabilities.” as well as the Abstract, “The mobile drive unit may be configured to traverse to locations associated with the identified system components. As the mobile drive unit traverses the inventory floor, it may connect to the separate networks that it comes into contact with.” Examiner Note: This demonstrates that tasks may require the robot to leave the network range in order to be completed and the robot will acquire instructions, traverse to the desired location, perform the task, and reconnect to a network upon returning within the communication range. It is further demonstrated that connection to a network is not required as the task instructions are communicated via the network as a part of the assignment process.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic edge network as taught by Griffin with the ability to determine latency and network information as taught by Banjade. Further incorporating the ability to connect/disconnect from a network as the device moves about the operating environment as taught by Brazeau into the control system as taught by Griffin would allow the system to operate in an environment where full network coverage is not available and to allow devices to connect to a variety of networks in order to communicate information on task assignments and progress. The ability to select which device/network based on latency and communication range would allow the system to select strong candidates at each point as well as to have a stronger more comprehensive network of devices interconnected.
Regarding claim 2, where all the limitations of claim 1 are discussed above, Griffin further teaches:
2. (previously presented) The computer-implemented method of claim 1, further comprising:
monitoring the subset of robotic edge devices while completing the task. (Paragraph 0059, "The server may also be communicatively coupled to computer(s) and/or device(s) that may be used to monitor and/or control robot 102 remotely. Communications unit 116 may also receive updates (e.g., firmware or data updates), data, statuses, commands, and other communications from a server for robot 102.")
Regarding claim 5, where all the limitations of claim 1 are discussed above, Griffin further teaches:
5. (previously presented) The computer-implemented method of claim 1, wherein the attributes further include:
physical capabilities of the robotic edge device; (Paragraph 0121, "FIG. 9 illustrates a data table 900 comprising data on a plurality of robots 102 on a robot network 304 and properties thereof, according to an exemplary embodiment. Each robot 102 may be assigned a robot ID distinguishing each of the plurality of robots 102. Each robot 102 may comprise a plurality of properties including, but not limited to, properties of a processing device of each robot 102 (e.g., clock rate, number of cores, etc.), a maximum speed of the robots 102 (e.g., meters per second, feet per second, etc.), a number of cameras on the robots 102, a functionality of the robots 102, and/or any other additional properties of the robots 102. Data table 900 may comprise N robots, wherein index N may be any non-zero integer number corresponding the number of robots 102 on the robot network 304. Similarly, index I may correspond to an arbitrary integer number of cameras on the N.sup.th robot 102 greater than or equal to zero. Data table 900 may be stored in a memory 132 of a cloud server 202 and may be accessed by a processing device 130 of the cloud server 202 to determine and distribute instructions to the plurality of robots 102 on the robot network 304.") and
geographic location. (Paragraph 0032,"As used herein, a robot network or network of robots may comprise a plurality of robots communicatively coupled to each other and/or coupled to an external cloud server. The plurality of robots may communicate data to other robots on the robot network and/or to an external cloud server. The plurality of robots on the robot network may include robots of different or the same functionalities. A robot network communicating with a server may comprise a plurality of robots on the robot network communicating with the server." and Paragraph 0056, "This odometry may include robot 102's position (e.g., where position may include robot's location, displacement and/or orientation, and may sometimes be interchangeable with the term pose as used herein) relative to the initial location. Such data may be stored in data structures, such as matrices, arrays, queues, lists, arrays, stacks, bags, etc. According to exemplary embodiments, the data structure of the sensor data may be called an image.")
Regarding claim 6, where all the limitations of claim 1 are discussed above, Griffin further teaches:
6. (Original) The computer-implemented method of claim 1, wherein each robotic edge device is selected from a group of robotic edge devices consisting of:
a reconnaissance robotic device;
a debris clearing robotic device;
an excavating robotic device;
a smart vehicle; and
an unmanned aerial vehicle. (Paragraph 0031, "As used herein, a robot may include mechanical and/or virtual entities configurable to carry out a complex series of tasks or actions autonomously. In some exemplary embodiments, robots may be machines that are guided and/or instructed by computer programs and/or electronic circuitry. In some exemplary embodiments, robots may include electro-mechanical components that are configurable for navigation, where the robot may move from one location to another. Such robots may include autonomous and/or semi-autonomous cars, floor cleaners, rovers, drones, planes, boats, carts, trams, wheelchairs, industrial equipment, stocking machines, mobile platforms, personal transportation devices (e.g., hover boards, SEGWAYS?, etc.), stocking machines, trailer movers, vehicles, and the like. Robots may also include any autonomous and/or semi-autonomous machine for transporting items, people, animals, cargo, freight, objects, luggage, and/or anything desirable from one location to another.")
Examiner Note: Robotic devices such as rovers and drones are well known in the field of robotics to be ideal for performing reconnaissance tasks in areas which are not ideal for human access. Griffin does not specifically call their system a reconnaissance system but it performs the tasks of reconnaissance which is demonstrated by at least the “data collection” tasks described. Similarly, Griffin does not specifically call out an excavating device. However, they do discuss the application of their work to industrial equipment which is known to include excavating devices.
Regarding claim 7, where all the limitations of claim 1 are discussed above, Griffin further teaches:
7. (Original) The computer-implemented method of claim 1, wherein the task is determined by analyzing data input from one or more Internet of Things (IoT) devices. (Paragraph 0032, "As used herein, a robot network or network of robots may comprise a plurality of robots communicatively coupled to each other and/or coupled to an external cloud server. The plurality of robots may communicate data to other robots on the robot network and/or to an external cloud server. The plurality of robots on the robot network may include robots of different or the same functionalities. A robot network communicating with a server may comprise a plurality of robots on the robot network communicating with the server." and Paragraph 0042, "According to at least one non-limiting exemplary embodiment, a method for collecting data, performing a task, and/or performing a computational function using a distributed robot network is disclosed. The method may include a cloud server receiving an operator input, generating an instruction to be executed by the robot network, and generating an operator output based on data collected during execution of the instruction by the robot network. The method may additionally include utilizing data collected by the robot network during execution of instructions to simultaneously respond to other operator inputs.)
Regarding claim 8, where all the limitations of claim 7 are discussed above, Griffin further teaches:
8. (Original) The computer-implemented method of claim 7, wherein the one or more IoT devices is selected from a group of IoT devices consisting of: a camera, a sensor, a computer. (Paragraph 0069, "External devices 206 may comprise user interface units, closed-circuit television (CCTV) cameras, other cloud servers 202, and/or any other type of device communicatively coupled to the cloud server 202. The processor 130 of the cloud server 202 may utilize data from the a respective one or more of the plurality of external devices 206 to determine individualized tasks to be performed by a respective one or more of the plurality of robots 102 coupled to the cloud server 202, and/or execute instructions in memory 132 based on the received data from the external devices 206. For example, an external device 206 may be a user interface, wherein data from the user interface may comprise a request for data, a physical task to be performed by the robots, and/or a request for a computation to be performed by the plurality of robots 102. The assigned individualized tasks to the robots 102 coupled to the cloud server 202 may enhance the efficiency of the cloud server 202 to respond to an input from the external device 206, as illustrated below in FIG. 6-8, as the work load required to respond to the input may be distributed among the plurality of robots 102.")
Regarding claim 10, where all the limitations of claim 9 are discussed above, Griffin further teaches:
10. (previously presented) The computer-implemented method of claim 1, wherein a second one of the physical activities requires that at least one of the subset of robotic edge devices moves one or more objects at the geographic location (Paragraph 0031, "Such robots may include autonomous and/or semi-autonomous cars, floor cleaners, rovers, drones, planes, boats, carts, trams, wheelchairs, industrial equipment, stocking machines, mobile platforms, personal transportation devices (e.g., hover boards, SEGWAYS?, etc.), stocking machines, trailer movers, vehicles, and the like. Robots may also include any autonomous and/or semi-autonomous machine for transporting items, people, animals, cargo, freight, objects, luggage, and/or anything desirable from one location to another.") using edge computation between each robotic edge device of the subset of robotic edge devices. (Paragraph 0069, " For example, an external device 206 may be a user interface, wherein data from the user interface may comprise a request for data, a physical task to be performed by the robots, and/or a request for a computation to be performed by the plurality of robots 102. The assigned individualized tasks to the robots 102 coupled to the cloud server 202 may enhance the efficiency of the cloud server 202 to respond to an input from the external device 206, as illustrated below in FIG. 6-8, as the work load required to respond to the input may be distributed among the plurality of robots 102.")
Regarding claim 11, Griffin further teaches:
11. (Currently Amended) A system comprising:
a processor; (Paragraph 0035, “As used herein, processor or processing device, microprocessor, and/or digital processor may include any type of digital processing device such as, without limitation, digital signal processors (“DSPs”), reduced instruction set computers (“RISC”), complex instruction set computer (“CISC”) processors, microprocessors, gate arrays (e.g., field programmable gate arrays (“FPGAs”)), programmable logic device (“PLDs”), reconfigurable computer fabrics (“RCFs”), array processors, secure microprocessors, specialized processors (e.g., neuromorphic processors), and application-specific integrated circuits (“ASICs”). Such digital processors may be contained on a single unitary integrated circuit die or distributed across multiple components.”) and
a computer-readable storage medium communicatively coupled to the processor and storing program instructions which, when executed by the processor, cause the processor to perform a method (Paragraph 0011, “Example embodiments disclosed herein are directed to methods, systems and non-transitory computer readable mediums that may have computer readable instructions stored thereon, that when executed by at least one processor or processing device, configure the at least one processing device to”) comprising:
identifying a plurality of robotic edge devices in a geographic location; (Paragraph 0032,"As used herein, a robot network or network of robots may comprise a plurality of robots communicatively coupled to each other and/or coupled to an external cloud server. The plurality of robots may communicate data to other robots on the robot network and/or to an external cloud server. The plurality of robots on the robot network may include robots of different or the same functionalities. A robot network communicating with a server may comprise a plurality of robots on the robot network communicating with the server." and Paragraph 0056, "This odometry may include robot 102's position (e.g., where position may include robot's location, displacement and/or orientation, and may sometimes be interchangeable with the term pose as used herein) relative to the initial location. Such data may be stored in data structures, such as matrices, arrays, queues, lists, arrays, stacks, bags, etc. According to exemplary embodiments, the data structure of the sensor data may be called an image.")
determining attributes for each robotic edge device of the plurality of robotic edge devices, (Paragraph 0121, "FIG. 9 illustrates a data table 900 comprising data on a plurality of robots 102 on a robot network 304 and properties thereof, according to an exemplary embodiment. Each robot 102 may be assigned a robot ID distinguishing each of the plurality of robots 102. Each robot 102 may comprise a plurality of properties including, but not limited to, properties of a processing device of each robot 102 (e.g., clock rate, number of cores, etc.), a maximum speed of the robots 102 (e.g., meters per second, feet per second, etc.), a number of cameras on the robots 102, a functionality of the robots 102, and/or any other additional properties of the robots 102. Data table 900 may comprise N robots, wherein index N may be any non-zero integer number corresponding the number of robots 102 on the robot network 304. Similarly, index I may correspond to an arbitrary integer number of cameras on the N.sup.th robot 102 greater than or equal to zero. Data table 900 may be stored in a memory 132 of a cloud server 202 and may be accessed by a processing device 130 of the cloud server 202 to determine and distribute instructions to the plurality of robots 102 on the robot network 304.") …
identifying a task to be performed at the geographic location by the plurality of robotic edge devices, the task including a plurality of physical activities; (Paragraph 0041, "According to at least one non-limiting exemplary embodiment, a system comprising a cloud server communicatively coupled to a robot network comprising a plurality of robots is disclosed. The cloud server may be configurable to receive an input from an operator and generate an output to the operator based on data gathered by the plurality of robots on the robot network. The cloud server may be further configurable to generate an instruction to be executed by the robot network, the instruction may configure the robot network to compute and/or collect data necessary to respond to the operator input." and Paragraph 0069, " For example, an external device 206 may be a user interface, wherein data from the user interface may comprise a request for data, a physical task to be performed by the robots, and/or a request for a computation to be performed by the plurality of robots 102. The assigned individualized tasks to the robots 102 coupled to the cloud server 202 may enhance the efficiency of the cloud server 202 to respond to an input from the external device 206, as illustrated below in FIG. 6-8, as the work load required to respond to the input may be distributed among the plurality of robots 102." Examiner Note: The limitation of “physical activities” under BRI may encompass activities such as travelling to a secondary location, moving components in order to survey the environment, grasping/placing objects etc.)
determining, based on the attributes, a subset of robotic edge devices that are capable of completing the task;
assigning the subset of robotic edge devices to complete the task (Paragraph 0011, "Example embodiments disclosed herein are directed to methods, systems and non-transitory computer readable mediums that may have computer readable instructions stored thereon, that when executed by at least one processor or processing device, configure the at least one processing device to, receive an operator input comprising instructions for a robot network, the robot network comprising a plurality of independently operable robots that are communicatively coupled to each other in an environment; transmit the instructions to at least a first sub-set of robots in the robot network such that the first sub-set of robots execute the instructions in the environment, the instructions comprising a plurality of tasks to be performed by the first sub-set of robots such that a respective task of the plurality of tasks is assigned to a respective robot of the first sub-set of robots based on bandwidth of the respective robot; receive data collected by the first sub-set of robots during simultaneous performance of the plurality of tasks by the first sub-set of robots in the environment; and generate an operator output based on the data collected by the first sub-set of robots.");
initiating the subset of robotic edge devices to complete the task; (Paragraph 0012, " The methods, systems and non-transitory computer readable mediums disclosed herein are further configurable to execute the computer readable instructions to, transmit the instructions to the first sub-set of robots in the robot network only if response to the operator input is not previously stored in the memory. Wherein, the transmission of the instructions to the first sub-set of robots configures the respective robot to navigate from a first location to a second location and collect data on one or more items at the second location, and transmit the data collected on the one or more items to the at least one controller. Further, wherein the transmission of the instructions to the first sub-set of robots configures the respective robot to, retrieve one or more items from a designated pick-up location and drop the one or more items at a designated drop-off location, and transmit data to the at least one processing device simultaneously as the one or more items are relocated from the pick-up location to the drop-off location.")
determining that a first one of the physical activities is located (Paragraph 0012, “The methods, systems and non-transitory computer readable mediums disclosed herein are further configurable to execute the computer readable instructions to, transmit the instructions to the first sub-set of robots in the robot network only if response to the operator input is not previously stored in the memory. Wherein, the transmission of the instructions to the first sub-set of robots configures the respective robot to navigate from a first location to a second location and collect data on one or more items at the second location, and transmit the data collected on the one or more items to the at least one controller. Further, wherein the transmission of the instructions to the first sub-set of robots configures the respective robot to, retrieve one or more items from a designated pick-up location and drop the one or more items at a designated drop-off location, and transmit data to the at least one processing device simultaneously as the one or more items are relocated from the pick-up location to the drop-off location.”) …
determining that a first one of the subset of robotic edge devices can perform the first one of the physical activities (Paragraph 0130, “According to at least one non-limiting exemplary embodiment, wherein data table 1000 is stored in memory 132 of a cloud server 202, the cloud server 202 may access the data table 1000 to distribute tasks to robots 102 in a robot network 304 best suited to perform the task. Similarly, the cloud server 202 may access the data table 1000 to distribute tasks to the robots 102 on the robot network to satisfy a plurality of instructions simultaneously. For example, instruction 1 and its comprising tasks may be assigned to a first set of robots 102, instruction 2 and its comprising tasks may be assigned to a second set of different robots 102, and so forth. Additionally, the cloud server 202 may utilize data from completed tasks during execution of some instructions to satisfy, at least in part, other instructions and their comprising tasks.”) …
Griffin does not specifically teach the device attributes being computation latency or communication range or assigning a task to a device which leaves the network to complete the task before reconnecting. However, Banjade, in the same field of endeavor of robotic edge devices, teaches:
… the attributes including edge computation latency and edge communication range; (Paragraph 0051, "In an embodiment, the selecting module 223 may be configured for selecting at least one edge device from a plurality of edge devices within a communication range of the IoT device based on network conditions and computational latency associated with the plurality of edge devices. In an embodiment, the identifying module 225 may be configured for identifying the preferred network for connecting the IoT device with the selected edge device based on available bandwidth and historical inference time records of a plurality of networks associated with the IoT device." also see Paragraph 0115, "Communications from any IoT device 902 may be passed along a convenient path between any of the IoT devices 902 to reach the gateways 904. In these networks, the number of interconnections provide substantial redundancy, allowing communications to be maintained, even with the loss of a number of IoT devices 902. Further, the use of a mesh network may allow IoT devices 902 that are very low power or located at a distance from infrastructure to be used, as the range to connect to another IoT device 902 may be much less than the range to connect to the gateways 904." which discusses that some devices communicate through the mesh due to having a smaller communication range. This demonstrates that the system is aware of the range of each device and may select a device based on this characteristic. Please also see paragraphs 0133 and 0026.) …
However, Brazeau, in the same field of endeavor of robotics, teaches:
… outside the edge communication range; (Paragraph 0016, “Embodiments herein are directed to an inventory system that includes mobile drive units and/or other robotic components managed via a series of local networks. Specifically, features herein are directed to a series of separate wireless local networks, each operated by various system components. Upon entering within transmission range of a wireless local network, a mobile drive unit may connect to the local network, identify the system component associated with the local network, determine if the mobile drive unit has been provided with any instructions directed to the system component, and execute one or more instructions related to the identified system component. A system component may be any resource or device that performs a function on behalf of the inventory system. By way of non-limiting example, a system component may comprise a robotic device, an input sensor, an inventory holder, a workstation, or any other suitable system resource.”) …
without requiring edge computation in parallel with performance of the first one of the physical activities; (Paragraph 0081, “FIG. 8 illustrates an example interaction between a computing device executing a management module, a mobile drive unit, and a computing device associated with a system component in accordance with at least some embodiments. In FIG. 8, a computing device executing a management module 802 may operate a first local network 804. As a mobile drive unit 806 enters the vicinity of the first local network (e.g., comes within transmission range of a wireless transmitter/receiver), the mobile drive unit 806 is discovered by the computing device 802. Upon discovery, the computing device 802 identifies the next task in a queue of tasks to be completed which the mobile drive unit 806 is capable of completing. In some embodiments, the computing device 802 generates instructions 808 (e.g., a task assignment) to be provided to each system component involved in the completion of the task. For example, the computing device 802 may generate instructions to be executed by the mobile drive unit 806 that include an initial route, inventory holder identifier, and workstation identifier. The computing device 802 may also generate instructions to be executed by a loader device that indicates the inventory holder identifier to be loaded onto the mobile drive unit 806. Additionally, the computing device 802 may generate instructions to be executed by a workstation computing device. In some embodiments, each of the generated instructions may be provided to the mobile drive unit 806, and may subsequently be distributed to each computing device 810 associated with each respective system component by the mobile drive unit 806 as it comes into proximity of that system component.” Please also see figures 1 and 8. This demonstrates that instructions are communicated but the robot is not in constant connection to the network at all points during the assigned task. Physical activities such as control of the system to move between locations and performance of retrieval tasks are performed while outside of the communication range of other system components.) …
causing the first one of the subset of robotic edge devices to move outside the edge communication range to perform the first one of the physical activities without being connected to a network and without edge computation or edge communication; and (Paragraphs 0017-0018, “In accordance with an embodiment, a mobile drive unit may approach a central authority to receive instructions. The central authority may identify a task that the mobile drive unit is capable of performing and may generate a task assignment based on the identified task.
The task assignment may comprise a set of instructions that cause the mobile drive unit to perform various actions with respect to various system components (e.g., robotic devices, input sensors, inventory holders, etc.). In some embodiments, one or more wireless local networks may be operated with respect to these various system components. The mobile drive unit may be configured (via the task assignment) to visit each of these system components in turn and execute the actions indicated in the task assignment. As the mobile drive unit enters within a proximity of the system component, it may connect to the wireless local network associated with that system component. Upon establishing a connection to the wireless local network, the mobile drive unit may transmit instructions to cause the system component to complete the action indicated in the task assignment with respect to that system component.” As well as Paragraphs 0023-0024, “In some embodiments, a task assignment may include an identification of an inventory holder 108, an identification of one or more items in the inventory holder 108 to be retrieved, and an identification of a workstation. The task assignment may include instructions that cause a mobile drive unit 104 to retrieve the identified inventory holder, bring the retrieved inventory holder to the identified workstation, and provide the indication of the one or more items to the workstation to be conveyed to an administrator. Upon receiving a task assignment from the computing device executing the management module 102, the mobile drive unit 104 may generate a route to an indicated inventory holder 108 and subsequently to an indicated workstation.
Upon arriving at the inventory holder 108, the mobile drive unit may verify that the inventory holder 108 is the indicated inventory holder based on an inventory holder identifier located on or near that inventory holder 108. The mobile drive unit 104 may then retrieve the inventory holder 108 or one or more items located within the inventory holder 108 in accordance with the task assignment. The mobile drive unit 104 may then move to the indicated workstation with the retrieved inventory holder 108.”)
causing the first one of the subset of robotic edge devices to move back inside the edge communication range upon determining completion of the first one of the physical activities. (Paragraph 0022, “In some embodiments, the computing device executing the management module 102 may discover the presence of a mobile drive unit 104. For example, the mobile drive unit 104 may connect to a private network when it is within range of the short range communication mechanism 106. Upon detecting that the mobile drive unit 104 has connected to the private network, the computing device including the management module 102 may be configured to receive a report on the status of the mobile drive unit 104, identify a current status of the mobile drive unit, identify an appropriate task assignment for the mobile drive unit, generate instructions to result in the completion of the task assignment by the mobile drive unit 104, and transmit the task assignment to the mobile drive unit 104. In some embodiments, the management module 102 may determine what task assignment is appropriate for a mobile drive unit based on the type and/or capabilities associated with the mobile drive unit. For example, the management module 102 may receive an identifier associated with the mobile drive unit and may determine, based on that identifier, a type and/or identity of the mobile drive unit. The management module 102 may then query a database of mobile drive units to identify the detected mobile drive unit and its capabilities.” as well as the Abstract, “The mobile drive unit may be configured to traverse to locations associated with the identified system components. As the mobile drive unit traverses the inventory floor, it may connect to the separate networks that it comes into contact with.” Examiner Note: This demonstrates that tasks may require the robot to leave the network range in order to be completed and the robot will acquire instructions, traverse to the desired location, perform the task, and reconnect to a network upon returning within the communication range. It is further demonstrated that connection to a network is not required as the task instructions are communicated via the network as a part of the assignment process.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic edge network as taught by Griffin with the ability to determine latency and network information as taught by Banjade. Further incorporating the ability to connect/disconnect from a network as the device moves about the operating environment as taught by Brazeau into the control system as taught by Griffin would allow the system to operate in an environment where full network coverage is not available and to allow devices to connect to a variety of networks in order to communicate information on task assignments and progress. The ability to select which device/network based on latency and communication range would allow the system to select strong candidates at each point as well as to have a stronger more comprehensive network of devices interconnected.
Regarding claim 12, where all the limitations of claim 11 are discussed above, Griffin further teaches:
12. (previously presented) The system of claim 11, wherein the method performed by the processor further comprises:
monitoring the subset of robotic edge devices while completing the task. (Paragraph 0059, "The server may also be communicatively coupled to computer(s) and/or device(s) that may be used to monitor and/or control robot 102 remotely. Communications unit 116 may also receive updates (e.g., firmware or data updates), data, statuses, commands, and other communications from a server for robot 102.")
Regarding claim 14, where all the limitations of claim 11 are discussed above, Griffin further teaches:
14. (previously presented) The system of claim 11, wherein the attributes further include:
physical capabilities of the robotic edge device; (Paragraph 0121, "FIG. 9 illustrates a data table 900 comprising data on a plurality of robots 102 on a robot network 304 and properties thereof, according to an exemplary embodiment. Each robot 102 may be assigned a robot ID distinguishing each of the plurality of robots 102. Each robot 102 may comprise a plurality of properties including, but not limited to, properties of a processing device of each robot 102 (e.g., clock rate, number of cores, etc.), a maximum speed of the robots 102 (e.g., meters per second, feet per second, etc.), a number of cameras on the robots 102, a functionality of the robots 102, and/or any other additional properties of the robots 102. Data table 900 may comprise N robots, wherein index N may be any non-zero integer number corresponding the number of robots 102 on the robot network 304. Similarly, index I may correspond to an arbitrary integer number of cameras on the N.sup.th robot 102 greater than or equal to zero. Data table 900 may be stored in a memory 132 of a cloud server 202 and may be accessed by a processing device 130 of the cloud server 202 to determine and distribute instructions to the plurality of robots 102 on the robot network 304.") and
geographic location. (Paragraph 0032,"As used herein, a robot network or network of robots may comprise a plurality of robots communicatively coupled to each other and/or coupled to an external cloud server. The plurality of robots may communicate data to other robots on the robot network and/or to an external cloud server. The plurality of robots on the robot network may include robots of different or the same functionalities. A robot network communicating with a server may comprise a plurality of robots on the robot network communicating with the server." and Paragraph 0056, "This odometry may include robot 102's position (e.g., where position may include robot's location, displacement and/or orientation, and may sometimes be interchangeable with the term pose as used herein) relative to the initial location. Such data may be stored in data structures, such as matrices, arrays, queues, lists, arrays, stacks, bags, etc. According to exemplary embodiments, the data structure of the sensor data may be called an image.")
Regarding claim 15, where all the limitations of claim 11 are discussed above, Griffin further teaches:
15. (Original) The system of claim 11, wherein each robotic edge device is selected from a group of robotic edge devices consisting of:
a reconnaissance robotic device;
a debris clearing robotic device;
an excavating robotic device;
a smart vehicle; and
an unmanned aerial vehicle. (Paragraph 0031, "As used herein, a robot may include mechanical and/or virtual entities configurable to carry out a complex series of tasks or actions autonomously. In some exemplary embodiments, robots may be machines that are guided and/or instructed by computer programs and/or electronic circuitry. In some exemplary embodiments, robots may include electro-mechanical components that are configurable for navigation, where the robot may move from one location to another. Such robots may include autonomous and/or semi-autonomous cars, floor cleaners, rovers, drones, planes, boats, carts, trams, wheelchairs, industrial equipment, stocking machines, mobile platforms, personal transportation devices (e.g., hover boards, SEGWAYS?, etc.), stocking machines, trailer movers, vehicles, and the like. Robots may also include any autonomous and/or semi-autonomous machine for transporting items, people, animals, cargo, freight, objects, luggage, and/or anything desirable from one location to another.")
Examiner Note: Robotic devices such as rovers and drones are well known in the field of robotics to be ideal for performing reconnaissance tasks in areas which are not ideal for human access. Griffin does not specifically call their system a reconnaissance system but it performs the tasks of reconnaissance which is demonstrated by at least the “data collection” tasks described. Similarly, Griffin does not specifically call out an excavating device. However, they do discuss the application of their work to industrial equipment which is known to include excavating devices.
Regarding claim 16, Griffin further teaches:
16. (Currently Amended) A computer program product comprising a computer-readable storage medium having program instructions embodied therewith, (Paragraph 0011, “Example embodiments disclosed herein are directed to methods, systems and non-transitory computer readable mediums that may have computer readable instructions stored thereon, that when executed by at least one processor or processing device, configure the at least one processing device to”) the program instructions executable by a processor (Paragraph 0035, “As used herein, processor or processing device, microprocessor, and/or digital processor may include any type of digital processing device such as, without limitation, digital signal processors (“DSPs”), reduced instruction set computers (“RISC”), complex instruction set computer (“CISC”) processors, microprocessors, gate arrays (e.g., field programmable gate arrays (“FPGAs”)), programmable logic device (“PLDs”), reconfigurable computer fabrics (“RCFs”), array processors, secure microprocessors, specialized processors (e.g., neuromorphic processors), and application-specific integrated circuits (“ASICs”). Such digital processors may be contained on a single unitary integrated circuit die or distributed across multiple components.”) to cause the processor to perform a method comprising:
identifying a plurality of robotic edge devices in a geographic location; (Paragraph 0032,"As used herein, a robot network or network of robots may comprise a plurality of robots communicatively coupled to each other and/or coupled to an external cloud server. The plurality of robots may communicate data to other robots on the robot network and/or to an external cloud server. The plurality of robots on the robot network may include robots of different or the same functionalities. A robot network communicating with a server may comprise a plurality of robots on the robot network communicating with the server." and Paragraph 0056, "This odometry may include robot 102's position (e.g., where position may include robot's location, displacement and/or orientation, and may sometimes be interchangeable with the term pose as used herein) relative to the initial location. Such data may be stored in data structures, such as matrices, arrays, queues, lists, arrays, stacks, bags, etc. According to exemplary embodiments, the data structure of the sensor data may be called an image.")
determining attributes for each robotic edge device of the plurality of robotic edge devices, (Paragraph 0121, "FIG. 9 illustrates a data table 900 comprising data on a plurality of robots 102 on a robot network 304 and properties thereof, according to an exemplary embodiment. Each robot 102 may be assigned a robot ID distinguishing each of the plurality of robots 102. Each robot 102 may comprise a plurality of properties including, but not limited to, properties of a processing device of each robot 102 (e.g., clock rate, number of cores, etc.), a maximum speed of the robots 102 (e.g., meters per second, feet per second, etc.), a number of cameras on the robots 102, a functionality of the robots 102, and/or any other additional properties of the robots 102. Data table 900 may comprise N robots, wherein index N may be any non-zero integer number corresponding the number of robots 102 on the robot network 304. Similarly, index I may correspond to an arbitrary integer number of cameras on the N.sup.th robot 102 greater than or equal to zero. Data table 900 may be stored in a memory 132 of a cloud server 202 and may be accessed by a processing device 130 of the cloud server 202 to determine and distribute instructions to the plurality of robots 102 on the robot network 304.") … ;
identifying a task to be performed at the geographic location by the plurality of robotic edge devices, the task including a plurality of physical activities; (Paragraph 0041, "According to at least one non-limiting exemplary embodiment, a system comprising a cloud server communicatively coupled to a robot network comprising a plurality of robots is disclosed. The cloud server may be configurable to receive an input from an operator and generate an output to the operator based on data gathered by the plurality of robots on the robot network. The cloud server may be further configurable to generate an instruction to be executed by the robot network, the instruction may configure the robot network to compute and/or collect data necessary to respond to the operator input." and Paragraph 0069, " For example, an external device 206 may be a user interface, wherein data from the user interface may comprise a request for data, a physical task to be performed by the robots, and/or a request for a computation to be performed by the plurality of robots 102. The assigned individualized tasks to the robots 102 coupled to the cloud server 202 may enhance the efficiency of the cloud server 202 to respond to an input from the external device 206, as illustrated below in FIG. 6-8, as the work load required to respond to the input may be distributed among the plurality of robots 102." Examiner Note: The limitation of “physical activities” under BRI may encompass activities such as travelling to a secondary location, moving components in order to survey the environment, grasping/placing objects etc.)
determining, based on the attributes, a subset of robotic edge devices that are capable of completing the task;
assigning the subset of robotic edge devices to complete the task (Paragraph 0011, "Example embodiments disclosed herein are directed to methods, systems and non-transitory computer readable mediums that may have computer readable instructions stored thereon, that when executed by at least one processor or processing device, configure the at least one processing device to, receive an operator input comprising instructions for a robot network, the robot network comprising a plurality of independently operable robots that are communicatively coupled to each other in an environment; transmit the instructions to at least a first sub-set of robots in the robot network such that the first sub-set of robots execute the instructions in the environment, the instructions comprising a plurality of tasks to be performed by the first sub-set of robots such that a respective task of the plurality of tasks is assigned to a respective robot of the first sub-set of robots based on bandwidth of the respective robot; receive data collected by the first sub-set of robots during simultaneous performance of the plurality of tasks by the first sub-set of robots in the environment; and generate an operator output based on the data collected by the first sub-set of robots.");
initiating the subset of robotic edge devices to complete the task; (Paragraph 0012, " The methods, systems and non-transitory computer readable mediums disclosed herein are further configurable to execute the computer readable instructions to, transmit the instructions to the first sub-set of robots in the robot network only if response to the operator input is not previously stored in the memory. Wherein, the transmission of the instructions to the first sub-set of robots configures the respective robot to navigate from a first location to a second location and collect data on one or more items at the second location, and transmit the data collected on the one or more items to the at least one controller. Further, wherein the transmission of the instructions to the first sub-set of robots configures the respective robot to, retrieve one or more items from a designated pick-up location and drop the one or more items at a designated drop-off location, and transmit data to the at least one processing device simultaneously as the one or more items are relocated from the pick-up location to the drop-off location.") and
determining that a first one of the physical activities is located (Paragraph 0012, “The methods, systems and non-transitory computer readable mediums disclosed herein are further configurable to execute the computer readable instructions to, transmit the instructions to the first sub-set of robots in the robot network only if response to the operator input is not previously stored in the memory. Wherein, the transmission of the instructions to the first sub-set of robots configures the respective robot to navigate from a first location to a second location and collect data on one or more items at the second location, and transmit the data collected on the one or more items to the at least one controller. Further, wherein the transmission of the instructions to the first sub-set of robots configures the respective robot to, retrieve one or more items from a designated pick-up location and drop the one or more items at a designated drop-off location, and transmit data to the at least one processing device simultaneously as the one or more items are relocated from the pick-up location to the drop-off location.”) …
determining that a first one of the subset of robotic edge devices can perform the first one of the physical activities (Paragraph 0130, “According to at least one non-limiting exemplary embodiment, wherein data table 1000 is stored in memory 132 of a cloud server 202, the cloud server 202 may access the data table 1000 to distribute tasks to robots 102 in a robot network 304 best suited to perform the task. Similarly, the cloud server 202 may access the data table 1000 to distribute tasks to the robots 102 on the robot network to satisfy a plurality of instructions simultaneously. For example, instruction 1 and its comprising tasks may be assigned to a first set of robots 102, instruction 2 and its comprising tasks may be assigned to a second set of different robots 102, and so forth. Additionally, the cloud server 202 may utilize data from completed tasks during execution of some instructions to satisfy, at least in part, other instructions and their comprising tasks.”) …
Griffin does not specifically teach the device attributes being computation latency or communication range or assigning a task to a device which leaves the network to complete the task before reconnecting. However, Banjade, in the same field of endeavor of robotic edge devices, teaches:
… the attributes including edge computation latency and edge communication range; (Paragraph 0051, "In an embodiment, the selecting module 223 may be configured for selecting at least one edge device from a plurality of edge devices within a communication range of the IoT device based on network conditions and computational latency associated with the plurality of edge devices. In an embodiment, the identifying module 225 may be configured for identifying the preferred network for connecting the IoT device with the selected edge device based on available bandwidth and historical inference time records of a plurality of networks associated with the IoT device." also see Paragraph 0115, "Communications from any IoT device 902 may be passed along a convenient path between any of the IoT devices 902 to reach the gateways 904. In these networks, the number of interconnections provide substantial redundancy, allowing communications to be maintained, even with the loss of a number of IoT devices 902. Further, the use of a mesh network may allow IoT devices 902 that are very low power or located at a distance from infrastructure to be used, as the range to connect to another IoT device 902 may be much less than the range to connect to the gateways 904." which discusses that some devices communicate through the mesh due to having a smaller communication range. This demonstrates that the system is aware of the range of each device and may select a device based on this characteristic. Please also see paragraphs 0133 and 0026.) …
However, Brazeau, in the same field of endeavor of robotics, teaches:
… outside the edge communication range; (Paragraph 0016, “Embodiments herein are directed to an inventory system that includes mobile drive units and/or other robotic components managed via a series of local networks. Specifically, features herein are directed to a series of separate wireless local networks, each operated by various system components. Upon entering within transmission range of a wireless local network, a mobile drive unit may connect to the local network, identify the system component associated with the local network, determine if the mobile drive unit has been provided with any instructions directed to the system component, and execute one or more instructions related to the identified system component. A system component may be any resource or device that performs a function on behalf of the inventory system. By way of non-limiting example, a system component may comprise a robotic device, an input sensor, an inventory holder, a workstation, or any other suitable system resource.”) …
without requiring edge computation in parallel with performance of the first one of the physical activities; (Paragraph 0081, “FIG. 8 illustrates an example interaction between a computing device executing a management module, a mobile drive unit, and a computing device associated with a system component in accordance with at least some embodiments. In FIG. 8, a computing device executing a management module 802 may operate a first local network 804. As a mobile drive unit 806 enters the vicinity of the first local network (e.g., comes within transmission range of a wireless transmitter/receiver), the mobile drive unit 806 is discovered by the computing device 802. Upon discovery, the computing device 802 identifies the next task in a queue of tasks to be completed which the mobile drive unit 806 is capable of completing. In some embodiments, the computing device 802 generates instructions 808 (e.g., a task assignment) to be provided to each system component involved in the completion of the task. For example, the computing device 802 may generate instructions to be executed by the mobile drive unit 806 that include an initial route, inventory holder identifier, and workstation identifier. The computing device 802 may also generate instructions to be executed by a loader device that indicates the inventory holder identifier to be loaded onto the mobile drive unit 806. Additionally, the computing device 802 may generate instructions to be executed by a workstation computing device. In some embodiments, each of the generated instructions may be provided to the mobile drive unit 806, and may subsequently be distributed to each computing device 810 associated with each respective system component by the mobile drive unit 806 as it comes into proximity of that system component.” Please also see figures 1 and 8. This demonstrates that instructions are communicated but the robot is not in constant connection to the network at all points during the assigned task. Physical activities such as control of the system to move between locations and performance of retrieval tasks are performed while outside of the communication range of other system components.) …
causing the first one of the subset of robotic edge devices to move outside the edge communication range to perform the first one of the physical activities without being connected to a network and without edge computation or edge communication; and (Paragraphs 0017-0018, “In accordance with an embodiment, a mobile drive unit may approach a central authority to receive instructions. The central authority may identify a task that the mobile drive unit is capable of performing and may generate a task assignment based on the identified task.
The task assignment may comprise a set of instructions that cause the mobile drive unit to perform various actions with respect to various system components (e.g., robotic devices, input sensors, inventory holders, etc.). In some embodiments, one or more wireless local networks may be operated with respect to these various system components. The mobile drive unit may be configured (via the task assignment) to visit each of these system components in turn and execute the actions indicated in the task assignment. As the mobile drive unit enters within a proximity of the system component, it may connect to the wireless local network associated with that system component. Upon establishing a connection to the wireless local network, the mobile drive unit may transmit instructions to cause the system component to complete the action indicated in the task assignment with respect to that system component.” As well as Paragraphs 0023-0024, “In some embodiments, a task assignment may include an identification of an inventory holder 108, an identification of one or more items in the inventory holder 108 to be retrieved, and an identification of a workstation. The task assignment may include instructions that cause a mobile drive unit 104 to retrieve the identified inventory holder, bring the retrieved inventory holder to the identified workstation, and provide the indication of the one or more items to the workstation to be conveyed to an administrator. Upon receiving a task assignment from the computing device executing the management module 102, the mobile drive unit 104 may generate a route to an indicated inventory holder 108 and subsequently to an indicated workstation.
Upon arriving at the inventory holder 108, the mobile drive unit may verify that the inventory holder 108 is the indicated inventory holder based on an inventory holder identifier located on or near that inventory holder 108. The mobile drive unit 104 may then retrieve the inventory holder 108 or one or more items located within the inventory holder 108 in accordance with the task assignment. The mobile drive unit 104 may then move to the indicated workstation with the retrieved inventory holder 108.”)
causing the first one of the subset of robotic edge devices to move back inside the edge communication range upon determining completion of the first one of the physical activities. (Paragraph 0022, “In some embodiments, the computing device executing the management module 102 may discover the presence of a mobile drive unit 104. For example, the mobile drive unit 104 may connect to a private network when it is within range of the short range communication mechanism 106. Upon detecting that the mobile drive unit 104 has connected to the private network, the computing device including the management module 102 may be configured to receive a report on the status of the mobile drive unit 104, identify a current status of the mobile drive unit, identify an appropriate task assignment for the mobile drive unit, generate instructions to result in the completion of the task assignment by the mobile drive unit 104, and transmit the task assignment to the mobile drive unit 104. In some embodiments, the management module 102 may determine what task assignment is appropriate for a mobile drive unit based on the type and/or capabilities associated with the mobile drive unit. For example, the management module 102 may receive an identifier associated with the mobile drive unit and may determine, based on that identifier, a type and/or identity of the mobile drive unit. The management module 102 may then query a database of mobile drive units to identify the detected mobile drive unit and its capabilities.” as well as the Abstract, “The mobile drive unit may be configured to traverse to locations associated with the identified system components. As the mobile drive unit traverses the inventory floor, it may connect to the separate networks that it comes into contact with.” Examiner Note: This demonstrates that tasks may require the robot to leave the network range in order to be completed and the robot will acquire instructions, traverse to the desired location, perform the task, and reconnect to a network upon returning within the communication range. It is further demonstrated that connection to a network is not required as the task instructions are communicated via the network as a part of the assignment process.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the robotic edge network as taught by Griffin with the ability to determine latency and network information as taught by Banjade. Further incorporating the ability to connect/disconnect from a network as the device moves about the operating environment as taught by Brazeau into the control system as taught by Griffin would allow the system to operate in an environment where full network coverage is not available and to allow devices to connect to a variety of networks in order to communicate information on task assignments and progress. The ability to select which device/network based on latency and communication range would allow the system to select strong candidates at each point as well as to have a stronger more comprehensive network of devices interconnected.
Regarding claim 17, where all the limitations of claim 16 are discussed above, Griffin further teaches:
17. (previously presented) The computer program product of claim 16, wherein the method performed by the processor further comprises:
monitoring the subset of robotic edge devices while completing the task. (Paragraph 0059, "The server may also be communicatively coupled to computer(s) and/or device(s) that may be used to monitor and/or control robot 102 remotely. Communications unit 116 may also receive updates (e.g., firmware or data updates), data, statuses, commands, and other communications from a server for robot 102.")
Regarding claim 19, where all the limitations of claim 16 are discussed above, Griffin further teaches:
19. (previously presented) The computer program product of claim 16, wherein the attributes further include:
physical capabilities of the robotic edge device; (Paragraph 0121, "FIG. 9 illustrates a data table 900 comprising data on a plurality of robots 102 on a robot network 304 and properties thereof, according to an exemplary embodiment. Each robot 102 may be assigned a robot ID distinguishing each of the plurality of robots 102. Each robot 102 may comprise a plurality of properties including, but not limited to, properties of a processing device of each robot 102 (e.g., clock rate, number of cores, etc.), a maximum speed of the robots 102 (e.g., meters per second, feet per second, etc.), a number of cameras on the robots 102, a functionality of the robots 102, and/or any other additional properties of the robots 102. Data table 900 may comprise N robots, wherein index N may be any non-zero integer number corresponding the number of robots 102 on the robot network 304. Similarly, index I may correspond to an arbitrary integer number of cameras on the N.sup.th robot 102 greater than or equal to zero. Data table 900 may be stored in a memory 132 of a cloud server 202 and may be accessed by a processing device 130 of the cloud server 202 to determine and distribute instructions to the plurality of robots 102 on the robot network 304."); and
geographic location. (Paragraph 0032,"As used herein, a robot network or network of robots may comprise a plurality of robots communicatively coupled to each other and/or coupled to an external cloud server. The plurality of robots may communicate data to other robots on the robot network and/or to an external cloud server. The plurality of robots on the robot network may include robots of different or the same functionalities. A robot network communicating with a server may comprise a plurality of robots on the robot network communicating with the server." and Paragraph 0056, "This odometry may include robot 102's position (e.g., where position may include robot's location, displacement and/or orientation, and may sometimes be interchangeable with the term pose as used herein) relative to the initial location. Such data may be stored in data structures, such as matrices, arrays, queues, lists, arrays, stacks, bags, etc. According to exemplary embodiments, the data structure of the sensor data may be called an image.")
Regarding claim 20, where all the limitations of claim 16 are discussed above, Griffin further teaches:
20. (Original) The computer program product of claim 16, wherein each robotic edge device is selected from a group of robotic edge devices consisting of:
a reconnaissance robotic device;
a debris clearing robotic device;
an excavating robotic device;
a smart vehicle; and
an unmanned aerial vehicle. (Paragraph 0031, "As used herein, a robot may include mechanical and/or virtual entities configurable to carry out a complex series of tasks or actions autonomously. In some exemplary embodiments, robots may be machines that are guided and/or instructed by computer programs and/or electronic circuitry. In some exemplary embodiments, robots may include electro-mechanical components that are configurable for navigation, where the robot may move from one location to another. Such robots may include autonomous and/or semi-autonomous cars, floor cleaners, rovers, drones, planes, boats, carts, trams, wheelchairs, industrial equipment, stocking machines, mobile platforms, personal transportation devices (e.g., hover boards, SEGWAYS?, etc.), stocking machines, trailer movers, vehicles, and the like. Robots may also include any autonomous and/or semi-autonomous machine for transporting items, people, animals, cargo, freight, objects, luggage, and/or anything desirable from one location to another.")
Examiner Note: Robotic devices such as rovers and drones are well known in the field of robotics to be ideal for performing reconnaissance tasks in areas which are not ideal for human access. Griffin does not specifically call their system a reconnaissance system but it performs the tasks of reconnaissance which is demonstrated by at least the “data collection” tasks described. Similarly, Griffin does not specifically call out an excavating device. However, they do discuss the application of their work to industrial equipment which is known to include excavating devices.
Conclusion
The Examiner has cited particular paragraphs or columns and line numbers in the referencesapplied to the claims above for the convenience of the Applicant. Although the specified citations arerepresentative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested of the Applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner. See MPEP 2141.02 [R-07.2015] VI. A prior art reference must be considered in its entirety, i.e., as a whole, including portions that would lead away from the claimed Invention. W.L. Gore & Associates, Inc. v. Garlock, Inc., 721 F.2d 1540, 220 USPQ 303 (Fed. Cir. 1983), cert, denied, 469 U.S. 851 (1984). See also MPEP §2123.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
/H.J.K./Examiner, Art Unit 3657
/ADAM R MOTT/Supervisory Patent Examiner, Art Unit 3657
Prosecution Insights