Detailed Action
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of the Application and Claims
This action is in reply to the application filed on 2/3/2025.
This communication is the first action on the merits.
IDS filed on 2/3/2025 is acknowledged and considered by the Examiner.
Claims 1-14 is/are currently pending and have been examined.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-14 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter.
Claim 1 (similarly 13, 14) recites, … executing the instructions to:
acquire space information, wherein the space information is information of a physical space;
acquire work specification information, wherein the work specification information is specification information of work performed by an … in the physical space;
determine, based on the space information and the work specification information, lacking information that is lacking with respect to the space information for planning execution of the work that is defined by the work specification information; and
present the lacking information that was determined to a requester of the work.
Analyzing under Step 2A, Prong 1:
The limitations regarding, …acquire space information, wherein the space information is information of a physical space; acquire work specification information, wherein the work specification information is specification information of work performed by an … in the physical space; determine, based on the space information and the work specification information, lacking information that is lacking with respect to the space information for planning execution of the work that is defined by the work specification information; and present the lacking information that was determined to a requester of the work…., under the broadest reasonable interpretation, can include a human using their mind and using pen and paper to perform the above identified limitations, therefore, the claims recite a mental process.
Further, …acquire space information, wherein the space information is information of a physical space; acquire work specification information, wherein the work specification information is specification information of work performed by an … in the physical space; determine, based on the space information and the work specification information, lacking information that is lacking with respect to the space information for planning execution of the work that is defined by the work specification information; and present the lacking information that was determined to a requester of the work…, are human observing work in work space and noting work space information required to perform work in the work space and reporting the lacking of work space information to a human requester, which are, managing personal behavior or relationships or interactions between people, therefore the claims recite certain methods of organizing human activities.
Accordingly, the claims recite a mental process, certain methods of organizing human activities, and thus, the claims are directed to an abstract idea under the first prong of Step 2A.
Analyzing under Step 2A, Prong 2:
This judicial exception is not integrated into a practical application under the second prong of Step 2A.
In particular, the claims recite the additional elements beyond the recited abstract idea identified under Step 2A, Prong 1, such as:
Claim 1, 13, 14: An information processing apparatus comprising: one or more memories storing instructions; and one or more processors, autonomously moving movable apparatus, A non-transitory computer-readable storage medium storing a computer program including instructions
, and pursuant to the broadest reasonable interpretation, as an ordered combination, each of the additional elements are computing elements recited at high level of generality implementing the abstract idea, and thus, are no more than applying the abstract idea with generic computer components.
Further, these additional elements generally link the abstract idea to a technical environment, namely the environment of a computer.
Additionally, with respect to, “…acquire…”, “…present…”, these elements do not add a meaningful limitations to integrate the abstract idea into a practical application because they are extra-solution activity, pre and post solution activity - i.e. data gathering – “…acquire…”, data output – “…present…”
Analyzing under Step 2B:
The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under Step 2B.
As noted above, the aforementioned additional elements beyond the recited abstract idea are not sufficient to amount to significantly more than the recited abstract idea because, as an order combination, the additional elements are no more than mere instructions to implement the idea using generic computer components (i.e. apply it).
Additionally, as an order combination, the additional elements append the recited abstract idea to well-understood, routine, and conventional activities in the field as individually evinced by the applicant’s own disclosure, as required by the Berkheimer Memo, in at least:
[0018] It should be noted that in the present embodiment, although an example of an autonomously moving cleaning robot is explained as a movable apparatus, the robot may be a robot that performs any kind of work. Furthermore, the present invention may also be applied to movable apparatuses such as drones or vehicles and the like that are capable of autonomous operation, in addition to robots.
[0094] It should be noted that FIG. 9 is a diagram showing a hardware configuration example of the information processing apparatus according to the First to Third Embodiments. Reference numeral 900 denotes an information processing apparatus. Reference numeral 901 denotes a CPU, 902 denotes a RAM, 903 denotes a storage unit serving as a storage medium such as an HDD or SSD, 904 denotes a communication unit, and 905 denotes a system bus.
[0095] The CPU 901 uses the RAM 902 serving as a work memory, executes an OS (Operating System) and various computer programs stored in the storage unit 903, and controls each part of the information processing apparatus via the system bus 905. Thereby, the information processing method shown in the flowcharts of FIG. 3 and FIG.8 can be executed.
[0096] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation to encompass all such modifications and equivalent structures and functions.
[0097] In addition, as a part or the whole of the control according to the embodiments, a computer program realizing the function of the embodiments described above may be supplied to the information processing apparatus and the like through a network or various storage media. Then, a computer (or a CPU, an MPU, or the like) of the information processing apparatus and the like may be configured to read and execute the program. In such a case, the program and the storage medium storing the program configure the present invention.
Furthermore, as an ordered combination, these elements amount to generic computer components receiving or transmitting data over a network, performing repetitive calculations, electronic record keeping, and storing and retrieving information in memory, which, as held by the courts, are well-understood, routine, and conventional. See MPEP 2106.05(d).
Moreover, the remaining elements of dependent claims do not transform the recited abstract idea into a patent eligible invention because these remaining elements merely recite further abstract limitations that provide nothing more than simply a narrowing of the abstract idea recited in the independent claims.
Looking at these limitations as an ordered combination adds nothing additional that is sufficient to amount to significantly more than the recited abstract idea because they simply provide instructions to use a generic arrangement of generic computer components to “apply” the recited abstract idea, perform insignificant extra-solution activity, and generally link the abstract idea to a technical environment. Thus, the elements of the claims, considered both individually and as an ordered combination, are not sufficient to ensure that the claim as a whole amounts to significantly more than the abstract idea itself. Since there are no limitations in these claims that transform the exception into a patent eligible application such that these claims amount to significantly more than the exception itself, claims 1-14 are rejected under 35 U.S.C. 101 as being directed to non-statutory subject matter.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 5-6, 13-14 is/are rejected under 35 U.S.C. 102 as being unpatentable by US Patent Publication to US20240142240A1to Afrouzi et al., (hereinafter referred to as “Afrouzi”).
As per Claim 1, Afrouzi teaches: An information processing apparatus comprising: one or more memories storing instructions; and one or more processors executing the instructions to: ([0007])
acquire space information, wherein the space information is information of a physical space; (in at least [0009] the application receives at least one input: associating a task to be performed by the robot with a particular zone within the map; associating a cleaning intensity with a particular zone within the map; associating a cleaning frequency or a cleaning schedule with a particular zone within the map; and designating an area within the map as an area the robot is to avoid. [0080] FIG. 1A illustrates an embodiment of the present invention where image sensor 100 mounted on robotic device 101 measures depth vectors 102 within a first field of view 103 of place 104. Image sensor 100 may, for example, be a LIDAR, LADAR, ultrasonic, camera, sonar, laser, or stereo depth measurement device or any other device or system capable of measuring depths or capturing data from which depth may be inferred. In some embodiments image sensor 100 is coupled with at least one laser emitter. A processor of robotic device 101 uses depth vectors to calculate the L2 norm and hence estimate depth from image sensor 100 to object 105. Referring to FIG. 1B, the processor translates estimated depths from depth vectors 102 taken within first field of view 103 into Cartesian coordinates corresponding to coordinate system 106 observed by image sensor 100 mounted on robotic device 101, wherein each coordinate represents a different location within place 104. To translate a depth estimate into a Cartesian coordinate, the processor iteratively checks each coordinate within coordinate system 106 observed by image sensor 100 until the coordinate coinciding with the depth estimate is identified. The coordinate with which the depth estimate coincides is identified as a perimeter of the place. In FIG. 1B, for example, a visual representation of coordinate system 106 observed by image sensor 100 is shown, whereby each coordinate corresponds to a cell representing a specific location within the environment. The coordinate cells coinciding with depth estimated from depth vectors 102 are the cells at which the depth vectors 102 terminate. All coordinates bound between perimeter coordinates and the limits of field of view 103 of image sensor 100 are treated as internal area.)
acquire work specification information, wherein the work specification information is specification information of work performed by an autonomously moving movable apparatus in the physical space; (in at least [0009] the application receives at least one input: associating a task to be performed by the robot with a particular zone within the map; associating a cleaning intensity with a particular zone within the map; associating a cleaning frequency or a cleaning schedule with a particular zone within the map; and designating an area within the map as an area the robot is to avoid. [0080] FIG. 1C, translated coordinates 107 are identified as perimeter, corresponding to depths estimated from depth vectors 102 measured within field of view 103. In this example, perimeter coordinates and internal area are shown visually in a grid, however, perimeter coordinates 107 could simply be stored in a matrix or finite ordered list. In this example, coordinates 107 identified as perimeter are colored black while coordinates between perimeter coordinates 107 and the limits of field of view 103 treated as internal area 108 are colored grey. Robotic device 101 begins to perform work within area 109 of place 104, corresponding to internal area 108 within coordinate system 106 observed by image sensor 100, following along movement path 110. The movement path shown is for illustrative purposes and may be structured differently. [0081] As robotic device 101 translates and rotates while performing work within internal area 109, image sensor 100 continuously measures vectors within successive fields of view. After capturing depth vectors within each new fields of view, the processor estimates depths corresponding to the depth vectors and translates depth estimates into Cartesian coordinates. The processor identifies Cartesian coordinates corresponding to depth estimates as perimeter and coordinates bounded between perimeter coordinates and limits of the field of view as internal area, alongside previously identified perimeter coordinates and internal area.)
determine, based on the space information and the work specification information, lacking information that is lacking with respect to the space information for planning execution of the work that is defined by the work specification information; and (in at least [0009] the application receives at least one input: associating a task to be performed by the robot with a particular zone within the map; associating a cleaning intensity with a particular zone within the map; associating a cleaning frequency or a cleaning schedule with a particular zone within the map; and designating an area within the map as an area the robot is to avoid. [0032] observed coordinate system of the robotic device and identifies them as perimeter, thereby expanding the discovered perimeters and internal areas with each new set of depth estimates. As the internal area within which the robotic device operates expands, new perimeters and internal areas of the place are reached and discovered. The robotic device continues to perform work within the continuously expanding internal area while the image sensor acquires data and the processor estimates depths and translates them into coordinates corresponding to the observed coordinate system of the robotic device, identifying them as perimeter of the place until at least a portion of the perimeter of the place is identified. [0033] empty spaces, perimeters, and spaces beyond the perimeter are captured by the image sensor, processed by the processor, stored in a memory, and expressed in an occupancy map by the processor, wherein each point within the occupancy map is assigned a status such as “unoccupied,” “occupied,” or “unknown.” Some embodiments may assign scores indicative of confidence in these classifications, e.g., based on a plurality of measurements that accumulate to increase confidence. By reducing the number of points processed and stored, however, the expressed computational costs are expected to also be reduced. [0041] processor of the robotic device verifies the accuracy of perimeters at any time during or after the process of finding the perimeter of the place. In embodiments where a perimeter is predicted by the map but not detected, the processor assigns corresponding data points on the map a lower confidence and in some instances the area is re-mapped with the approach above in response. In some embodiments, mapping the perimeters of the place is complete after the robotic device has made contact with all perimeters and confirmed that the locations at which contact with each perimeter was made coincides with the locations of corresponding perimeters in the map. In some embodiments, a conservative coverage algorithm is executed to cover the internal areas of the place before the processor of the robotic device checks if the observed perimeters in the map coincide with the true perimeters of the place. This ensures more area is covered before the robotic device faces challenging areas such as obstacles. In some embodiments, the processor uses this method to establish ground truth by determining the difference between the location of the perimeter coordinate and the actual location of the perimeter. In some embodiments, the processor uses a separate map to keep track of new perimeters discovered, thereby creating another map. The processor may merge two maps using different methods, such as the intersection or union of two maps. For example, in some embodiments, the processor may apply the union of two maps to create an extended map of the place with areas that may have been undiscovered in the first map and/or the second map. In some embodiments, the processor creates a second map on top of a previously created map in a layered fashion, resulting in additional areas of the place that may have not been recognized in the original map. Such methods are used, for example, in cases where areas are separated by movable obstacles that may have prevented the image sensor and processor of the robotic device from determining the full map of the place and in some cases, completing an assigned task. For example, a soft curtain may act as a movable object that appears as a wall in a first map. In this case, the processor creates a second map on top of the previously created first map in a layered fashion to add areas to the original map that may have not been previously discovered. The processor of the robotic device then recognizes (e.g., determine) the area behind the curtain that may be important (e.g., warrant adjusting a route based on) in completing an assigned task. [0060] the processor of the robotic device fails to localize the robotic device and is unaware of its current location. In such instances, the processor resets localization and mapping and begins creating a new map, visiting each area (or room) for mapping, beginning with the room in which localization was lost. Since localization was lost, the processor is unaware of the initial starting point (i.e., the location of the docking station) to which it is to return after completing a work session. In such embodiments, after completing the work session the robotic device visits each area (or room) of the environment and rotates 360 degrees until a receiver of the robotic device detects an IR light emitted by a transmitter of the docking station signaling the robotic device to return to its initial starting position (i.e., the docking station).)
present the lacking information that was determined to a requester of the work. (in at least [0009] the robot to drive within the workspace to form a map with perimeters that correspond with physical perimeters of the workspace while obtaining, with the one or more processors, second data indicative of movement of the robot as the robot drives within the workspace, the second data being based on at least output of a second sensor of different type than the first sensor; and forming, with the one or more processors, the map of the workspace based on at least some of the first data; wherein: the map of the workspace expands as new data of the workspace are obtained and until all perimeters of the workspace are included in the map; an application of a communication device paired with the robot displays the map; the application receives at least one input: associating a task to be performed by the robot with a particular zone within the map; associating a cleaning intensity with a particular zone within the map; associating a cleaning frequency or a cleaning schedule with a particular zone within the map; and designating an area within the map as an area the robot is to avoid. [0033] empty spaces, perimeters, and spaces beyond the perimeter are captured by the image sensor, processed by the processor, stored in a memory, and expressed in an occupancy map by the processor, wherein each point within the occupancy map is assigned a status such as “unoccupied,” “occupied,” or “unknown.” Some embodiments may assign scores indicative of confidence in these classifications, e.g., based on a plurality of measurements that accumulate to increase confidence. By reducing the number of points processed and stored, however, the expressed computational costs are expected to also be reduced. For example, points beyond the perimeters may not always be processed. [0053] the processor uses coordinates corresponding with internal areas for path planning. For example, the processor can consecutively order coordinates corresponding with internal areas such that the movement path of the robotic device follows along coordinates in an ordered manner. As new coordinates correspond with newly discovered internal areas, the processor updates the movement path of the robotic device such that the newly discovered internal areas are covered. [0058] with each successive working session, the map generated during that session may be compiled with maps generated from prior work cycles. In some embodiments, the compiled maps may generate a comprehensive map of all the maps previously generated. In some embodiments, the comprehensive map may contain data suggestive of trends in the work environment. In some embodiments, for example, trends regarding obstacles such as the type of obstacle encountered, location of obstacle encountered, how often obstacle is encountered, the date and or time obstacle was encountered and the like data may be utilized for the planning of a work session or navigational route. In some embodiments, once an aggregate map is generated, a robotic device may be controlled or directed to navigate or operate in locations in a work area based on the aggregate map. In some embodiments, various navigation patterns and operational functions based on the aggregate map generated may be envisioned. In some embodiments, a robotic device may be controlled to navigate and operate based on historical data, such as, for example, by prioritizing operation in areas of a map where low interference has occurred such as, for example, a low likelihood of encountering an obstacle. In some embodiments, a robotic device may be controlled or directed to navigate or operate in areas based on preferences set in prior working sessions. In some embodiments, a robotic device may be controlled or directed to navigate or operate in areas based on work surface type, such as, for example, a robotic device being controlled to operate at a higher rate of navigation speed on hard work surface types such as tile. In some embodiments, a robotic device may be controlled or directed to navigate or operate in a portion of a work area rather than to operate in an entire work area. In some embodiments, a robotic device may be controlled or directed to operate in a first area prior to operating in a second area. In some embodiments, preferences may be set with regards to a working session such that scheduling, operational functions to be performed in the work session, and the like are preset rather than the robotic device utilizing data from prior work cycles to predict and enact an operations plan. In some embodiments, machine learning may be utilized by the robotic device, such that data from prior work sessions is utilized to predict and enact a work session based on data collected from prior work cycles. For example, a robotic device may utilize data pertaining to locations operated in, how successful a navigational route was, obstacles encountered, types of obstacles encountered, types of work surface operated on, scheduling information, preferences utilized in prior working sessions, whether multiple robotic devices were utilized, battery efficiency, and the like information. [0064] A user may select with a cursor, pointer, stylus, mouse, the user's finger, a button or buttons, a keyboard, or other input devices any portion of the workspace and select one or more settings to be applied to the area. In another example, a user may set a cleaning mode for different sections. In setting a cleaning mode, the user may, for example, set a service condition; a service type; a service parameter; or a service frequency. Service condition indicates whether an area is to be serviced or not, and embodiments may determine whether to service an area based on a specified service condition in memory. Thus, a regular service condition may indicate that the area is to be serviced in accordance with service parameters like those described below. In contrast, a no service condition may indicate that the area is to be excluded from service (e.g., cleaning). A service type indicates what kind of cleaning is to occur. For example, a hard (e.g. non-absorbent) surface may receive a mopping service (or vacuuming service followed by a mopping service in a service sequence), while a carpeted service may receive a vacuuming service.)
As per Claim 2, Afrouzi teaches: The information processing apparatus according to claim 1,
wherein the work specification information includes information that indicates a type of the work and information that indicates a region in which the work is executed from among the space information acquired. (in at least [0009] the application receives at least one input: associating a task to be performed by the robot with a particular zone within the map; associating a cleaning intensity with a particular zone within the map; associating a cleaning frequency or a cleaning schedule with a particular zone within the map; and designating an area within the map as an area the robot is to avoid. [0064] A user may select with a cursor, pointer, stylus, mouse, the user's finger, a button or buttons, a keyboard, or other input devices any portion of the workspace and select one or more settings to be applied to the area. In another example, a user may set a cleaning mode for different sections. In setting a cleaning mode, the user may, for example, set a service condition; a service type; a service parameter; or a service frequency. Service condition indicates whether an area is to be serviced or not, and embodiments may determine whether to service an area based on a specified service condition in memory. Thus, a regular service condition may indicate that the area is to be serviced in accordance with service parameters like those described below. In contrast, a no service condition may indicate that the area is to be excluded from service (e.g., cleaning). A service type indicates what kind of cleaning is to occur. For example, a hard (e.g. non-absorbent) surface may receive a mopping service (or vacuuming service followed by a mopping service in a service sequence), while a carpeted service may receive a vacuuming service.)
As per Claim 5, Afrouzi teaches: The information processing apparatus according to claim 1, wherein the one or more processors further execute the instructions to: perform at least one of
processing that calculates a time difference between a time at which the space information was acquired and an execution start time of the work, and processing that calculates a space information difference between the space information and the space information at a time when the movable apparatus measured the space information; and determine a reward to be granted to the requester based on at least one of the time difference and the space information difference. (in at least [0041] perimeters may be visited to check for accuracy before visiting all internal areas. For example, perimeters in a first room may be visited after coverage of internal areas in the first room and before coverage in a second room. Depending on the situation, the processor of the robotic device verifies the accuracy of perimeters at any time during or after the process of finding the perimeter of the place. In embodiments where a perimeter is predicted by the map but not detected, the processor assigns corresponding data points on the map a lower confidence and in some instances the area is re-mapped with the approach above in response. In some embodiments, mapping the perimeters of the place is complete after the robotic device has made contact with all perimeters and confirmed that the locations at which contact with each perimeter was made coincides with the locations of corresponding perimeters in the map. In some embodiments, a conservative coverage algorithm is executed to cover the internal areas of the place before the processor of the robotic device checks if the observed perimeters in the map coincide with the true perimeters of the place. This ensures more area is covered before the robotic device faces challenging areas such as obstacles. In some embodiments, the processor uses this method to establish ground truth by determining the difference between the location of the perimeter coordinate and the actual location of the perimeter. In some embodiments, the processor uses a separate map to keep track of new perimeters discovered, thereby creating another map. The processor may merge two maps using different methods, such as the intersection or union of two maps. For example, in some embodiments, the processor may apply the union of two maps to create an extended map of the place with areas that may have been undiscovered in the first map and/or the second map. In some embodiments, the processor creates a second map on top of a previously created map in a layered fashion, resulting in additional areas of the place that may have not been recognized in the original map. Such methods are used, for example, in cases where areas are separated by movable obstacles that may have prevented the image sensor and processor of the robotic device from determining the full map of the place and in some cases, completing an assigned task. For example, a soft curtain may act as a movable object that appears as a wall in a first map. In this case, the processor creates a second map on top of the previously created first map in a layered fashion to add areas to the original map that may have not been previously discovered. The processor of the robotic device then recognizes (e.g., determine) the area behind the curtain that may be important (e.g., warrant adjusting a route based on) in completing an assigned task.)
As per Claim 6, Afrouzi teaches: The information processing apparatus according to claim 5,
wherein the reward is made larger as at least one of the time difference and the space information difference becomes smaller. (in at least [0040] the processor uses a Markov Decision Process (MDP) consisting of a sequence of states and actions followed by rewards. Actions are taken to transition from one state to another and, after transitioning to each new state, the processor assigns a reward. For a sequence of states and actions, the processor calculates a net reward as the sum of rewards received for the sequence of states and actions, with future rewards discounted. The expected net reward for the execution of a sequence of states and actions is given by a state-action value function. The processor is configured to find the optimal state-action value function by identifying the sequence of states and actions, including coordinate system resolution to use in executing the actions, with highest net reward. Since multiple actions can be taken from each state, the goal of the processor is to also find an optimal policy that indicates the action, including coordinate system resolution to use in executing the action, from each state with the highest reward value. For example, if the robotic device is observed to bump into an obstacle while executing an action using a low-resolution coordinate system, the processor calculates a lower reward than when the robotic device completes the same action free of any collision using a high-resolution coordinate system, assuming collisions with obstacles reduces the reward achieved. If this is repeated over time, the processor eventually derives a policy to execute that particular action using a high-resolution coordinate system as it achieves higher reward. In some embodiments, as the robotic device executes more actions using high- and low-resolution coordinate systems over time, data is gathered on the reward assigned to each state and action, the action including the coordinate system resolution used in executing the action. In some embodiments, the processor compares the reward received for executing an action from one state to another using a high-resolution coordinate system and executing the same action using a low-resolution coordinate system. Over time the processor determines a policy that maximizes the net reward. In some embodiments, the sequence of states and actions corresponds to the states visited and actions taken (including the resolution of the coordinate system used in completing each action) while, for example, executing a work session from start to finish. Over time, as more states are visited and different actions from each state are evaluated, the system will converge to find the most optimal action (including the resolution of the coordinate system used in completing each action) to take from each state thereby forming an optimal policy. Further, as different sequences of states and actions are evaluated over time, the system will converge to the most optimal sequence of states and actions. For example, consider the states visited and actions taken from each state while cleaning a room using a high and low-resolution coordinate system. If the robotic device has multiple encounters with obstacles and coverage time is increased while executing actions from different states during the cleaning session the processor calculates a lower net reward (assuming collisions and cleaning time are factors in determining the reward value) than when completing the cleaning session collision free using a high-resolution coordinate system. If this is continuously observed over time, the processor derives a policy to use a high-resolution coordinate system for the particular actions taken while cleaning the room. In this example, only two levels of coordinate system resolution are considered for illustrative purposes, however, the processor can consider a greater number of different resolution levels. [0041] a conservative coverage algorithm is executed to cover the internal areas of the place before the processor of the robotic device checks if the observed perimeters in the map coincide with the true perimeters of the place. This ensures more area is covered before the robotic device faces challenging areas such as obstacles. In some embodiments, the processor uses this method to establish ground truth by determining the difference between the location of the perimeter coordinate and the actual location of the perimeter. In some embodiments, the processor uses a separate map to keep track of new perimeters discovered, thereby creating another map. The processor may merge two maps using different methods, such as the intersection or union of two maps. For example, in some embodiments, the processor may apply the union of two maps to create an extended map of the place with areas that may have been undiscovered in the first map and/or the second map. In some embodiments, the processor creates a second map on top of a previously created map in a layered fashion, resulting in additional areas of the place that may have not been recognized in the original map. Such methods are used, for example, in cases where areas are separated by movable obstacles that may have prevented the image sensor and processor of the robotic device from determining the full map of the place and in some cases, completing an assigned task. For example, a soft curtain may act as a movable object that appears as a wall in a first map. In this case, the processor creates a second map on top of the previously created first map in a layered fashion to add areas to the original map that may have not been previously discovered. The processor of the robotic device then recognizes (e.g., determine) the area behind the curtain that may be important (e.g., warrant adjusting a route based on) in completing an assigned task.)
As per Claim 13, 14 for A method (see at least Afrouzi [0007]) and A non-transitory computer-readable storage medium (see at least Afrouzi [0007]), respectively, substantially recite the subject matter of Claim 1 and are rejected based on the same reasoning and rationale.
Claim Rejections – 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 3-4, 7-12 is/are rejected under 35 U.S.C. 103 as being unpatentable by US Patent Publication to US20240142240A1to Afrouzi et al., (hereinafter referred to as “Afrouzi”) in view of US Patent Publication to US20240391112A1 to Hirai et al., (hereinafter referred to as “Hirai”)
As per Claim 3, Afrouzi teaches: The information processing apparatus according to claim 1,
wherein the space includes a …. (in at least [0057] a robotic device may be ordered to skip operational functions in a work area. For example, a button on a user interface of the robotic device, or an application of a communications device that is paired with the robotic device may be utilized for sending a command to a processor of the robotic device to skip operations in a given work area. An example of a communications device includes but is not limited to: a smart phone, smart watch, laptop, tablet, remote control, or the like. In some embodiments, if a robotic device enters a work area, the robotic device may be commanded to leave the work area. For example, when individuals are in a work area, a robotic device may attempt to operate in the same work area which may be burdensome or frustrating for the individuals, and the robotic device may be commanded to leave the work area. [0065] the perimeter map accessed for modification and/or updating is encrypted using a Public Key Infrastructure wherein a private key and digital signature is required to access the perimeter map. In some embodiments, the information is encrypted with a public key to protect against intercept.)
Although implied, Afrouzi does not expressly disclose the following limitations, which however, are taught by Hirai,
…concealed region in which concealment processing has been applied to a region that includes information related to confidentiality…(in at least [0278] the intermediary unit 212 blurs, replaces with an image of CG, or erases a concealed area designated by the requester to make the concealed area invisible, thereby generating work image data in which confidential information is concealed. The concealed area is, for example, an area including a product before release, a design drawing, a design image, and the like, an area including a non-public manufacturing processing [0280] the robot 14 may process the work image data, and transmit the processed work image data to the management server 15. As a result, risk of leakage of confidential information can be further reduced. Furthermore, for example, the management server 15 or the robot 14 may also perform processing of concealing confidential information on the work sensor data.)
At the time the invention was filed, it would have been obvious for one of ordinary skill in the art to have modified the teachings of Afrouzi as taught by Hirai, with a reasonable expectation of success if arriving at the claimed invention. One of ordinary skill in the art would have been motivated to make this modification to the teachings of Afrouzi with the motivation of, …an operator or a requester who requests work desires appropriate matching between the operator and the robot.…connecting a requester (business operator) and an operator (remote worker) of remote operation. The service platform includes a remote robot operation system and a remote operation matching system….enables remote operation, remote monitoring, device management, and the like of each robot, and enables guarantee of work quality….the operator can improve quality of work by performing remote operation according to the assist information.…the attention of the operator can be attracted, and the quality of the work can be improved….determines whether or not the work status has been improved, on the basis of a result of the processing…traceability is improved in investigation of a cause of a defective product…quality of work performed by the remote operation is improved, and a satisfaction level of the requester is improved. Furthermore, since the work according to the ability of the operator is matched, the satisfaction level is improved….another operator can assist the operator or urge the operator to improve the work. Furthermore, for example, in a case where a plurality of operators performs work in order, an operator of a downstream process can grasp a work delay of an upstream process in advance and take appropriate measures… in order to effectively utilize a robot that can be remotely operated…, as recited in Hirai.
As per Claim 4, Afrouzi teaches: The information processing apparatus according to claim 3,
wherein the determination of lacking information comprises, searching for an overlapping region between the region in which the work is executed and the …, and in a case in which the overlapping region exists, determining whether information types necessary for executing the work are included in the overlapping region, and determining information types … as types of the deficient information. (in at least [0042] processor uses overlapping coordinates to verify the accuracy of the identified perimeter. Assuming the frame rate of the image sensor is fast enough to capture more than one frame of data in the time it takes the robotic device to rotate the width of the frame, a portion of data captured within each field of view will overlap with a portion of data captured within the preceding field of view. In some embodiments, the processor verifies accuracy of perimeter coordinates by assigning a vote (although other point systems can be used, such as providing a reward or assigning an arbitrary numerical value or symbol) to each coordinate identified as perimeter each time a depth estimated from data captured in a separate field of view translates to the same coordinate, thereby overlapping with it. In some embodiments, coordinates with increased number votes are considered to be more accurate. Multiple number of votes arise from multiple sets of data overlapping with one another and increase the accuracy in the predicted perimeter. In some embodiments, the processor ignores coordinates with a number of votes below a specified threshold. [0053] the processor uses coordinates corresponding with internal areas for path planning. For example, the processor can consecutively order coordinates corresponding with internal areas such that the movement path of the robotic device follows along coordinates in an ordered manner. As new coordinates correspond with newly discovered internal areas, the processor updates the movement path of the robotic device such that the newly discovered internal areas are covered. [0055] a robotic device may attempt to make a determination as to whether or not it has visited a work area previously. In some embodiments, utilizing a localization module, SLAM module, or the like a robotic device may localize and place itself within an internal map of the working environment to determine its location. In some embodiments, sensors, cameras or the like may capture information of the working environment in determining whether or not a work area has been previously visited. For example, cameras of a robotic device may capture images of a working environment such as obstacles, the meeting points of floors and walls and the like. A processor of a robotic device may extrapolate features in these images in order to determine a layout of a work area. In some embodiments, a robotic device may have a database of images captured of the work environment and the processor of the robotic device may identify common features between the images captured and those in the database in order to determine if a work area has previously been visited. Alternatively, LIDAR may capture features of the working environment, and utilize the data captured to identify features of a work area based on data from prior work cycles. In some embodiments, a robotic device may identify features in a given work area, such as a wall pattern and attempt to identify whether or not those are features that were identified previously in the same working session. For example, if a robotic device has identified a unique characteristic of a wall, if the robotic device returns and identifies this characteristic in the wall area, a processor of the robotic device may determine that it has already operated in this location. In some embodiments, a unique characteristic may appear slightly different when identified a second time, and a processor of a robotic device may need to manipulate the data of the unique characteristic captured in order to determine whether or not the characteristic accurately matches what was previously captured when determining whether or not a work area has previously been operated in. For example, a unique characteristic of a work area may be captured initially up close, but at a later time, the same characteristic may be captured from farther away, or from a different sensor type, or from a different angle, or the like. In such a situation a processor of the robotic device may need to manipulate the data captured such as, for example, enlarging an image captured from farther away to attempt to match with the image captured up close, or by, for example rotating the image if the images were captured from different angles or the like to determine if the images match each other. In some embodiments, features of a work environment may be captured which will aid in determining whether or not a work environment has been previously visited, such as, for example, identifying particular obstacle as having been already encountered. As another example, a robotic device may encounter a transition from one work surface type to another such as a wood flooring surface to a thick pile carpet surface, and based on where the transition is in a work area in relation to where the robotic device is located in an internal map of the work environment, a processor of the robotic device may determine that it has already visited this location. [0071] the robotic device may, for example, use the perimeter map to autonomously navigate the place during operation, e.g., accessing the perimeter map to determine that a candidate route is blocked by an obstacle denoted in the perimeter map, to select a route with a route-finding algorithm from a current point to a target point, or the like. In some embodiments, the perimeter map is stored in memory for future use. Storage of the perimeter map may be in temporary memory such that a stored perimeter map is only available during an operational session or in more permanent forms of memory such that the perimeter map is available at the next session or startup. In some embodiments, the perimeter map is further processed to identify rooms and other segments. In some embodiments, a new perimeter map is constructed at each use, or an extant perimeter map is updated based on newly acquired data.)
Although implied, Afrouzi does not expressly disclose the following limitations, which however, are taught by Hirai,
…concealed region… (in at least [0278] the intermediary unit 212 blurs, replaces with an image of CG, or erases a concealed area designated by the requester to make the concealed area invisible, thereby generating work image data in which confidential information is concealed. The concealed area is, for example, an area including a product before release, a design drawing, a design image, and the like, an area including a non-public manufacturing processing [0280] the robot 14 may process the work image data, and transmit the processed work image data to the management server 15. As a result, risk of leakage of confidential information can be further reduced. Furthermore, for example, the management server 15 or the robot 14 may also perform processing of concealing confidential information on the work sensor data.)
determining information types not included therein as types of the deficient information (in at least [0278] the intermediary unit 212 blurs, replaces with an image of CG, or erases a concealed area designated by the requester to make the concealed area invisible, thereby generating work image data in which confidential information is concealed. The concealed area is, for example, an area including a product before release, a design drawing, a design image, and the like, an area including a non-public manufacturing processing)
The reason and rationale to combine Afrouzi and Hirai is the same as recited above.
As per Claim 7, Afrouzi teaches: The information processing apparatus according to claim 1, wherein the one or more processors further execute the instructions to
determine a reward to be granted …. (in at least [0040] the processor evaluates the performance of the robotic device in executing actions using different coordinate system resolutions. In some embodiments, the processor uses a Markov Decision Process (MDP) consisting of a sequence of states and actions followed by rewards. Actions are taken to transition from one state to another and, after transitioning to each new state, the processor assigns a reward. For a sequence of states and actions, the processor calculates a net reward as the sum of rewards received for the sequence of states and actions, with future rewards discounted. The expected net reward for the execution of a sequence of states and actions is given by a state-action value function.)
Although implied, Afrouzi does not expressly disclose the following limitations, which however, are taught by Hirai,
…a reward to be granted to the requester… (in at least [0062] connecting a requester (business operator) and an operator (remote worker) of remote operation. The service platform includes a remote robot operation system and a remote operation matching system. [0116] the evaluation unit 214 updates an evaluation score (described later) of the operator on the basis of evaluation by the requester. For example, the evaluation unit 214 determines a reward to be given to the operator on the basis of the evaluation on the work content by the operator and the evaluation on the work by the requester. [0158] The operator evaluates the requester and the work requested by the requester, and notifies the management server 15 of an evaluation result. [0460] The evaluation by the operator on the current work includes, for example, evaluation on the requester and evaluation on the requested work. The evaluation on the requester includes, for example, evaluation on a response, a reward, and the like of the requester. The evaluation on the requested work includes, for example, an evaluation on a difficulty level, a work environment, and the like of the work.)
The reason and rationale to combine Afrouzi and Hirai is the same as recited above.
As per Claim 8, Afrouzi teaches: The information processing apparatus according to claim 7,
wherein the reward is determined based on a difference between work execution time calculated when planning execution of the work and actual work time taken when the movable apparatus executed the work. (in at least [0040] the processor uses a Markov Decision Process (MDP) consisting of a sequence of states and actions followed by rewards. Actions are taken to transition from one state to another and, after transitioning to each new state, the processor assigns a reward. For a sequence of states and actions, the processor calculates a net reward as the sum of rewards received for the sequence of states and actions, with future rewards discounted. The expected net reward for the execution of a sequence of states and actions is given by a state-action value function. The processor is configured to find the optimal state-action value function by identifying the sequence of states and actions, including coordinate system resolution to use in executing the actions, with highest net reward. Since multiple actions can be taken from each state, the goal of the processor is to also find an optimal policy that indicates the action, including coordinate system resolution to use in executing the action, from each state with the highest reward value. For example, if the robotic device is observed to bump into an obstacle while executing an action using a low-resolution coordinate system, the processor calculates a lower reward than when the robotic device completes the same action free of any collision using a high-resolution coordinate system, assuming collisions with obstacles reduces the reward achieved. If this is repeated over time, the processor eventually derives a policy to execute that particular action using a high-resolution coordinate system as it achieves higher reward. In some embodiments, as the robotic device executes more actions using high- and low-resolution coordinate systems over time, data is gathered on the reward assigned to each state and action, the action including the coordinate system resolution used in executing the action. In some embodiments, the processor compares the reward received for executing an action from one state to another using a high-resolution coordinate system and executing the same action using a low-resolution coordinate system. Over time the processor determines a policy that maximizes the net reward. In some embodiments, the sequence of states and actions corresponds to the states visited and actions taken (including the resolution of the coordinate system used in completing each action) while, for example, executing a work session from start to finish. Over time, as more states are visited and different actions from each state are evaluated, the system will converge to find the most optimal action (including the resolution of the coordinate system used in completing each action) to take from each state thereby forming an optimal policy. Further, as different sequences of states and actions are evaluated over time, the system will converge to the most optimal sequence of states and actions. For example, consider the states visited and actions taken from each state while cleaning a room using a high and low-resolution coordinate system. If the robotic device has multiple encounters with obstacles and coverage time is increased while executing actions from different states during the cleaning session the processor calculates a lower net reward (assuming collisions and cleaning time are factors in determining the reward value) than when completing the cleaning session collision free using a high-resolution coordinate system. If this is continuously observed over time, the processor derives a policy to use a high-resolution coordinate system for the particular actions taken while cleaning the room. In this example, only two levels of coordinate system resolution are considered for illustrative purposes, however, the processor can consider a greater number of different resolution levels.)
As per Claim 9, Afrouzi teaches: The information processing apparatus according to claim 7,
wherein the reward is determined based on a … between an amount of work calculated when execution of the work was planned and an actual amount of work performed by the movable apparatus. (in at least [0040] the processor uses a Markov Decision Process (MDP) consisting of a sequence of states and actions followed by rewards. Actions are taken to transition from one state to another and, after transitioning to each new state, the processor assigns a reward. For a sequence of states and actions, the processor calculates a net reward as the sum of rewards received for the sequence of states and actions, with future rewards discounted. The expected net reward for the execution of a sequence of states and actions is given by a state-action value function. The processor is configured to find the optimal state-action value function by identifying the sequence of states and actions, including coordinate system resolution to use in executing the actions, with highest net reward. Since multiple actions can be taken from each state, the goal of the processor is to also find an optimal policy that indicates the action, including coordinate system resolution to use in executing the action, from each state with the highest reward value. For example, if the robotic device is observed to bump into an obstacle while executing an action using a low-resolution coordinate system, the processor calculates a lower reward than when the robotic device completes the same action free of any collision using a high-resolution coordinate system, assuming collisions with obstacles reduces the reward achieved. If this is repeated over time, the processor eventually derives a policy to execute that particular action using a high-resolution coordinate system as it achieves higher reward. In some embodiments, as the robotic device executes more actions using high- and low-resolution coordinate systems over time, data is gathered on the reward assigned to each state and action, the action including the coordinate system resolution used in executing the action. In some embodiments, the processor compares the reward received for executing an action from one state to another using a high-resolution coordinate system and executing the same action using a low-resolution coordinate system. Over time the processor determines a policy that maximizes the net reward. In some embodiments, the sequence of states and actions corresponds to the states visited and actions taken (including the resolution of the coordinate system used in completing each action) while, for example, executing a work session from start to finish. Over time, as more states are visited and different actions from each state are evaluated, the system will converge to find the most optimal action (including the resolution of the coordinate system used in completing each action) to take from each state thereby forming an optimal policy. Further, as different sequences of states and actions are evaluated over time, the system will converge to the most optimal sequence of states and actions. For example, consider the states visited and actions taken from each state while cleaning a room using a high and low-resolution coordinate system. If the robotic device has multiple encounters with obstacles and coverage time is increased while executing actions from different states during the cleaning session the processor calculates a lower net reward (assuming collisions and cleaning time are factors in determining the reward value) than when completing the cleaning session collision free using a high-resolution coordinate system. If this is continuously observed over time, the processor derives a policy to use a high-resolution coordinate system for the particular actions taken while cleaning the room. In this example, only two levels of coordinate system resolution are considered for illustrative purposes, however, the processor can consider a greater number of different resolution levels. [0057] a robotic device may utilize historical data with regards to prior work operations when planning an operational session. For example, in some embodiments, for a mobile robotic cleaning device, an operational session set by the robotic device may utilize prior historical data with regards to the level of debris previously cleaned in each work area in determining which work areas should be cleaned first.)
Although implied, Afrouzi does not expressly disclose the following limitations, which however, are taught by Hirai,
…a difference between an amount of work calculated when execution of the work was planned and an actual amount of work performed…(in at least [0452] the management server 15 determines a reward on the basis of the evaluation on the work content and the evaluation by the requester, and updates the evaluation on the operator. [0453] in a case where a work achievement rate is less than 100%, that is, in a case where the work requested to the operator has not been completed, the evaluation unit 214 reduces the reward to be given to the operator from the reward presented at the time of requesting the work, on the basis of the work achievement rate. [0454] in a case where the achievement rate of the work is 100%, the evaluation unit 214 determines the reward on the basis of the evaluation by the requester on the work. For example, in a case where the evaluation by the requester is 100% or less, the evaluation unit 214 determines the reward to be given to the operator as the reward presented at the time of requesting the work. For example, in a case where the evaluation by the requester exceeds 100%, the evaluation unit 214 determines the reward to be given to the operator as a reward of 120% of the reward presented at the time of requesting the work.)
The reason and rationale to combine Afrouzi and Hirai is the same as recited above.
As per Claim 10, Afrouzi teaches: The information processing apparatus according to claim 7,
wherein the reward is determined according to an amount or ratio of the lacking information. (in at least [0040] the processor uses a Markov Decision Process (MDP) consisting of a sequence of states and actions followed by rewards. Actions are taken to transition from one state to another and, after transitioning to each new state, the processor assigns a reward. For a sequence of states and actions, the processor calculates a net reward as the sum of rewards received for the sequence of states and actions, with future rewards discounted. The expected net reward for the execution of a sequence of states and actions is given by a state-action value function. The processor is configured to find the optimal state-action value function by identifying the sequence of states and actions, including coordinate system resolution to use in executing the actions, with highest net reward. Since multiple actions can be taken from each state, the goal of the processor is to also find an optimal policy that indicates the action, including coordinate system resolution to use in executing the action, from each state with the highest reward value. For example, if the robotic device is observed to bump into an obstacle while executing an action using a low-resolution coordinate system, the processor calculates a lower reward than when the robotic device completes the same action free of any collision using a high-resolution coordinate system, assuming collisions with obstacles reduces the reward achieved. If this is repeated over time, the processor eventually derives a policy to execute that particular action using a high-resolution coordinate system as it achieves higher reward. In some embodiments, as the robotic device executes more actions using high- and low-resolution coordinate systems over time, data is gathered on the reward assigned to each state and action, the action including the coordinate system resolution used in executing the action. In some embodiments, the processor compares the reward received for executing an action from one state to another using a high-resolution coordinate system and executing the same action using a low-resolution coordinate system. Over time the processor determines a policy that maximizes the net reward. In some embodiments, the sequence of states and actions corresponds to the states visited and actions taken (including the resolution of the coordinate system used in completing each action) while, for example, executing a work session from start to finish. Over time, as more states are visited and different actions from each state are evaluated, the system will converge to find the most optimal action (including the resolution of the coordinate system used in completing each action) to take from each state thereby forming an optimal policy. Further, as different sequences of states and actions are evaluated over time, the system will converge to the most optimal sequence of states and actions. For example, consider the states visited and actions taken from each state while cleaning a room using a high and low-resolution coordinate system. If the robotic device has multiple encounters with obstacles and coverage time is increased while executing actions from different states during the cleaning session the processor calculates a lower net reward (assuming collisions and cleaning time are factors in determining the reward value) than when completing the cleaning session collision free using a high-resolution coordinate system. If this is continuously observed over time, the processor derives a policy to use a high-resolution coordinate system for the particular actions taken while cleaning the room. In this example, only two levels of coordinate system resolution are considered for illustrative purposes, however, the processor can consider a greater number of different resolution levels.)
As per Claim 11, Afrouzi teaches: The information processing apparatus according to claim 7,
wherein the reward is determined according to a ratio of a … from among a work target region. (in at least [0040] the processor uses a Markov Decision Process (MDP) consisting of a sequence of states and actions followed by rewards. Actions are taken to transition from one state to another and, after transitioning to each new state, the processor assigns a reward. For a sequence of states and actions, the processor calculates a net reward as the sum of rewards received for the sequence of states and actions, with future rewards discounted. The expected net reward for the execution of a sequence of states and actions is given by a state-action value function. The processor is configured to find the optimal state-action value function by identifying the sequence of states and actions, including coordinate system resolution to use in executing the actions, with highest net reward. Since multiple actions can be taken from each state, the goal of the processor is to also find an optimal policy that indicates the action, including coordinate system resolution to use in executing the action, from each state with the highest reward value. For example, if the robotic device is observed to bump into an obstacle while executing an action using a low-resolution coordinate system, the processor calculates a lower reward than when the robotic device completes the same action free of any collision using a high-resolution coordinate system, assuming collisions with obstacles reduces the reward achieved. If this is repeated over time, the processor eventually derives a policy to execute that particular action using a high-resolution coordinate system as it achieves higher reward. In some embodiments, as the robotic device executes more actions using high- and low-resolution coordinate systems over time, data is gathered on the reward assigned to each state and action, the action including the coordinate system resolution used in executing the action. In some embodiments, the processor compares the reward received for executing an action from one state to another using a high-resolution coordinate system and executing the same action using a low-resolution coordinate system. Over time the processor determines a policy that maximizes the net reward. In some embodiments, the sequence of states and actions corresponds to the states visited and actions taken (including the resolution of the coordinate system used in completing each action) while, for example, executing a work session from start to finish. Over time, as more states are visited and different actions from each state are evaluated, the system will converge to find the most optimal action (including the resolution of the coordinate system used in completing each action) to take from each state thereby forming an optimal policy. Further, as different sequences of states and actions are evaluated over time, the system will converge to the most optimal sequence of states and actions. For example, consider the states visited and actions taken from each state while cleaning a room using a high and low-resolution coordinate system. If the robotic device has multiple encounters with obstacles and coverage time is increased while executing actions from different states during the cleaning session the processor calculates a lower net reward (assuming collisions and cleaning time are factors in determining the reward value) than when completing the cleaning session collision free using a high-resolution coordinate system. If this is continuously observed over time, the processor derives a policy to use a high-resolution coordinate system for the particular actions taken while cleaning the room. In this example, only two levels of coordinate system resolution are considered for illustrative purposes, however, the processor can consider a greater number of different resolution levels. [0041] a conservative coverage algorithm is executed to cover the internal areas of the place before the processor of the robotic device checks if the observed perimeters in the map coincide with the true perimeters of the place. This ensures more area is covered before the robotic device faces challenging areas such as obstacles. In some embodiments, the processor uses this method to establish ground truth by determining the difference between the location of the perimeter coordinate and the actual location of the perimeter. In some embodiments, the processor uses a separate map to keep track of new perimeters discovered, thereby creating another map. The processor may merge two maps using different methods, such as the intersection or union of two maps. For example, in some embodiments, the processor may apply the union of two maps to create an extended map of the place with areas that may have been undiscovered in the first map and/or the second map. In some embodiments, the processor creates a second map on top of a previously created map in a layered fashion, resulting in additional areas of the place that may have not been recognized in the original map. Such methods are used, for example, in cases where areas are separated by movable obstacles that may have prevented the image sensor and processor of the robotic device from determining the full map of the place and in some cases, completing an assigned task. For example, a soft curtain may act as a movable object that appears as a wall in a first map. In this case, the processor creates a second map on top of the previously created first map in a layered fashion to add areas to the original map that may have not been previously discovered. The processor of the robotic device then recognizes (e.g., determine) the area behind the curtain that may be important (e.g., warrant adjusting a route based on) in completing an assigned task.)
Although implied, Afrouzi does not expressly disclose the following limitations, which however, are taught by Hirai,
…concealed region… (in at least [0278] the intermediary unit 212 blurs, replaces with an image of CG, or erases a concealed area designated by the requester to make the concealed area invisible, thereby generating work image data in which confidential information is concealed. The concealed area is, for example, an area including a product before release, a design drawing, a design image, and the like, an area including a non-public manufacturing processing [0280] the robot 14 may process the work image data, and transmit the processed work image data to the management server 15. As a result, risk of leakage of confidential information can be further reduced. Furthermore, for example, the management server 15 or the robot 14 may also perform processing of concealing confidential information on the work sensor data.)
The reason and rationale to combine Afrouzi and Hirai is the same as recited above.
As per Claim 12, Afrouzi teaches: The information processing apparatus according to claim 8,
wherein the reward is made larger as the difference becomes smaller. (in at least [0040] the processor uses a Markov Decision Process (MDP) consisting of a sequence of states and actions followed by rewards. Actions are taken to transition from one state to another and, after transitioning to each new state, the processor assigns a reward. For a sequence of states and actions, the processor calculates a net reward as the sum of rewards received for the sequence of states and actions, with future rewards discounted. The expected net reward for the execution of a sequence of states and actions is given by a state-action value function. The processor is configured to find the optimal state-action value function by identifying the sequence of states and actions, including coordinate system resolution to use in executing the actions, with highest net reward. Since multiple actions can be taken from each state, the goal of the processor is to also find an optimal policy that indicates the action, including coordinate system resolution to use in executing the action, from each state with the highest reward value. For example, if the robotic device is observed to bump into an obstacle while executing an action using a low-resolution coordinate system, the processor calculates a lower reward than when the robotic device completes the same action free of any collision using a high-resolution coordinate system, assuming collisions with obstacles reduces the reward achieved. If this is repeated over time, the processor eventually derives a policy to execute that particular action using a high-resolution coordinate system as it achieves higher reward. In some embodiments, as the robotic device executes more actions using high- and low-resolution coordinate systems over time, data is gathered on the reward assigned to each state and action, the action including the coordinate system resolution used in executing the action. In some embodiments, the processor compares the reward received for executing an action from one state to another using a high-resolution coordinate system and executing the same action using a low-resolution coordinate system. Over time the processor determines a policy that maximizes the net reward. In some embodiments, the sequence of states and actions corresponds to the states visited and actions taken (including the resolution of the coordinate system used in completing each action) while, for example, executing a work session from start to finish. Over time, as more states are visited and different actions from each state are evaluated, the system will converge to find the most optimal action (including the resolution of the coordinate system used in completing each action) to take from each state thereby forming an optimal policy. Further, as different sequences of states and actions are evaluated over time, the system will converge to the most optimal sequence of states and actions. For example, consider the states visited and actions taken from each state while cleaning a room using a high and low-resolution coordinate system. If the robotic device has multiple encounters with obstacles and coverage time is increased while executing actions from different states during the cleaning session the processor calculates a lower net reward (assuming collisions and cleaning time are factors in determining the reward value) than when completing the cleaning session collision free using a high-resolution coordinate system. If this is continuously observed over time, the processor derives a policy to use a high-resolution coordinate system for the particular actions taken while cleaning the room. In this example, only two levels of coordinate system resolution are considered for illustrative purposes, however, the processor can consider a greater number of different resolution levels.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PO HAN (Max) LEE whose telephone number is (571) 272-3821. The examiner can normally be reached on Monday - Thursday, 9 AM-6:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Rutao Wu can be reached on (571) 272-6045. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PO HAN LEE/Primary Examiner, Art Unit 3623
Prosecution Insights