DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s amendments have overcome the objection to claim 6, therefore, the claim objection has been withdrawn.
Applicant’s amendments have removed the claims from the scope of 35 USC 112(f) therefore the previous claim interpretation under 35 USC 112(f) has been withdrawn.
Applicant’s amendments have overcome the rejection of claims 1-11 under 35 USC 101 therefore the claim rejection under 35 USC 101 has been withdrawn.
Applicant’s arguments with respect to the rejection(s) of claim(s) 1-11 under 35 USC 102 and 35 USC 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, Applicant’s amendments to the claims changed the scope of the claims thereby necessitating a new ground(s) of rejection view of newly found prior art references.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1-4, 7, and 10-11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kranski et al. (US 20220314434 A1, hereinafter Kranski) in view of Tee et al. (US 20230330859 A1, hereinafter Tee).
Regarding claim 1, Kranski discloses an autonomous control system (Fig. 1, robot system 102) comprising:
a processor (Fig. 5, CPU 260) configured to:
acquire state data of a robot, visual data of the robot, and tactile data of the robot (at least as in paragraph 0034, wherein “Sensors of a robot system 102 may output sensor data”; at least as in paragraph 0043, wherein “Various sensors, such as tactile or force sensors or strain sensors or pressure sensors, along with cameras, inertial measurement units, and the like may output sensor data corresponding to movements and interactions of components of the arm with itself or other objects. Sensor data may be collected from various image or distance sensors, which may be processed, such as by encoder models, to output vectors indicative of position of the arm (or members thereof) within the environment and other environmental data. For example, as the robot moves towards an object, sensor data including images showing the environment around the robot, data indicating positions of legs and arms of the robot, etc.”; at least as in paragraph 0044, wherein “Sensor data may include position data from servo motors or stepper motors indicating the reported positions of one or more part of the robot, the reported positions of one or more parts of the robot relative to other parts of the robot, battery level, power consumption, motor current, or a variety of other information associated with state of the robot. Sensor data may include information obtained from a motor position sensor of the robot (e.g., located in arm, member, joint, or other part of the robot system 102), a touch sensor located in a part of the robot system 102 (e.g., a finger of the robot system 102), or a motor current sensor of the robot”); and
decide on an action of the robot capable of accomplishing a task given to the robot on the basis of the state data, the visual data, and the tactile data (at least as in paragraph 0068, wherein “a CPU 260 may execute a control model”; at least as in paragraph 0075, wherein “a CPU 260, which may execute one or more control models that may cause the robot to perform an action”; at least as in paragraph 0030, wherein “A trained control model 116A of the robot system 102A thus may account for (e.g., learn to accommodate) properties of the robot system 102A for which it generates instructions to perform robot control actions based on robot state, such as to cause the robot to complete a task”),
wherein the processor generates first compressed data having a smaller number of dimensions than data (at least as in paragraph 0038, wherein “Each (or at least some, such as upstream encoders) of the encoder models may transform relatively high-dimensional outputs of a robot's sensor suite into lower-dimensional vector representations”; at least as in paragraph 0041, wherein “the plurality of channels of sensor data may be transformed into embedding vectors within different sub-spaces of the latent embedding space by a first set of encoder models coupled to the sensors”; at least as in paragraph 0067, wherein “sensor layer 240 may thus include a plurality of sensors 240A-E, which may include one or more computer vision sensors (e.g., various cameras, LiDAR, etc.), proximity sensors (e.g., ultrasonic, etc.), tactile or force sensors or strain sensors or pressure sensors, inertial measurement units, and the like, among other sources of feedback data, like servos, stepper motors, actuators and the like”; at least as in paragraph 0069, wherein “encoder 250F may ingest sensor feedback data from upstream encoder models (e.g., 250D, 250E) as shown, or from sensors (e.g., 240A-E) of the sensor layer 240. Each encoder 250 may perform dimensionality reduction on inputs, but the amount of reduction may vary, such as whether an encoder is performing reduction on encoder outputs, sensor outputs, a combination thereof, and the type of data”; at least as in paragraph 0070, wherein “an encoder 250A may receive inputs from two or more sensors 240A, 240B”; at least as in paragraph 0073, wherein “one or more layers of encoder models may be implemented by hardware machine-learning accelerators”; at least as in paragraph 0076, wherein “ML Accelerator 250A may ingest sensor data from sensors 240A and 240B … execute an encoder model that generates a latent-space embedding based on the combined sensor data 240A and 240B … execute a convolutional neural network or a vision transformer to output a vector indicative of a slice of input data received from sensor 240A and 240B with in the latent space. The output by the ML accelerator 250A may thus be of a lower dimensionality”; at least as in paragraph 0135, wherein “the first accelerator may obtain first sensor data from a first subset of sensors, like two or more sensors of the robot [and] transform inputs received via outputs of the first subset of sensors into a first sub-space representation that accounts for properties sensed by the first subset of sensors of the robot”; therefore, Kranski teaches wherein an encoder dimensionally reduces and combines data from a vision sensor and a tactile sensor into a lower-dimensional vector),
wherein the processor generates second compressed data having a smaller number of dimensions than the tactile data by dimensionally compressing the tactile data using a (at least as in paragraph 0069, wherein “an encoder 250B that receives input from a touch sensor matrix may reduce dimensionality of received inputs by 10× or 100”; therefore, Kranski teaches wherein an encoder dimensionally reduces touch sensor data),
wherein the processor decides on the action on the basis of combined state data obtained by combining the state data, the first compressed data, and the second compressed data into one using a policy network (at least as in paragraph 0068, wherein “downstream encoder model 250F need not ingest each channel of sensor data directly, but rather may ingest representations of the sensor data from those channels that are output by upstream encoders”; at least as in paragraph 0069, wherein “Each encoder 250 may perform dimensionality reduction on inputs, but the amount of reduction may vary, such as whether an encoder is performing reduction on encoder outputs, sensor outputs, a combination thereof, and the type of data”; at least as in paragraph 0070, wherein “an encoder 250E may receive inputs from one or more encoders 250C (which receives inputs from one or more sensors 240D) and one or more sensors 240E”; at least as in paragraph 0079, wherein “a first accelerator may encode depth/distance information (e.g., from a LiDAR sensor), a second accelerator may encode object localization/detection data (e.g., from a camera), and a third encoder may combine the depth/distance information with the object localization/detection data without encountering the sensor information directly”; at least as in paragraph 0044, wherein “Sensor data may include position data from servo motors or stepper motors indicating the reported positions of one or more part of the robot, the reported positions of one or more parts of the robot relative to other parts of the robot, battery level, power consumption, motor current, or a variety of other information associated with state of the robot”; therefore, the encoder may combine the robot state data/information, the lower-dimensional vector of vision and tactile sensor data, and the dimensionally reduced touch sensor data; at least as in paragraph 0030, wherein “A trained control model 116A of the robot system 102A thus may account for (e.g., learn to accommodate) properties of the robot system 102A for which it generates instructions to perform robot control actions based on robot state, such as to cause the robot to complete a task”; at least as in paragraph 0075, wherein “a downstream encoder, like a HW ML Layer-3 ML Accelerator 250F may take input from one or more intermediate (e.g., Layer-2) accelerators 250D-E prior to providing output to a CPU 260, which may execute one or more control models that may cause the robot to perform an action based on the input it receives from the ML accelerator 250F”),
wherein the processor inputs the combined state data into the policy network (at least as in paragraph 0141, “the process may control the robot via one or more control models executed by one or more processor based on the latent-space representation. For example, a robot control model may include a reinforcement learning model trained at least in part via a reinforcement learning process, and the reinforcement learning model may take, as input, outputs of one or more encoder models. The encoder models executed by one or more ML Accelerators may simplify the input parameter space of the reinforcement learning model, which, due to complexity may be executed on a general purposed central processing unit. Reduction of the number of input parameters, for example, may reduce latency of model execution over a stream of input data”) and
wherein the processor generates a command that controls an actuator of the robot based on an action variable indicating the action that is output by the policy network in response to input of the combined state data (at least as in paragraph 0138, “In a step 425, the process may include controlling the robot based on the space embeddings. In some examples, a processor may use the combination of sub-space embeddings to control the robot”; at least as in paragraph 0141, “the process may control the robot via one or more control models executed by one or more processor based on the latent-space representation”).
Kranski does not explicitly disclose obtained by combining the visual data and the tactile data by fusing and dimensionally compressing the visual data and the tactile data using a first encoder… using a second encoder.
However, Tee, in the same field of endeavor of robot control using a combination of vision and tactile data, specifically discloses “obtained by combining the visual data and the tactile data by fusing and dimensionally compressing the visual data and the tactile data using a first encoder, (at least as in paragraph 0175, “The system 1400 comprises a first spiking neural network, SNN, encoder 1402 configured for encoding an event based output of a vision sensor 1404 into individual vision modality spiking representations with a first output size; a second SNN encoder 1406 configured for encoding an event based output of a tactile sensor 1408 into individual tactile modality spiking representations with a second output size; a combination layer 1410 configured for merging the vision modality spiking representations and the tactile modality spiking representations; and a task SNN 1412 configured to receive the merged vision modality spiking representations and tactile modality spiking representations and output vision-tactile modality spiking representations with a third output size for classification”; at least as in paragraph 0182, “At step 1506, the vision modality spiking representations and the tactile modality spiking representations are merged, using a combination layer. At step 1508, using a task SNN to receive the merged vision modality spiking representations and tactile modality spiking representations and to output vision-tactile modality spiking representations with a third output size for classification, using a task SNN to receive the concatenated vision modality spiking representations and tactile modality spiking representations”)…
using a second encoder (at least as in paragraph 0182, “At step 1504, an event-based output of a tactile sensor is encoded, using a second SNN encoder, into individual tactile modality spiking representations with a second output size”).”
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Kranski, to include Tee's teaching of encoding and merging visual and tactile data, since Tee teaches wherein encoding and merging of the data reduces computational costs and latency thus improving the effectiveness of a reasonable hardware setup.
Regarding claim 2, in view of the above combination of Kranski and Tee, Kranski further discloses the autonomous control system according to claim 1,
wherein the processor acquires depth image data generated by a camera that images a body of the robot and a target of the task as the visual data and acquires data in which a contact force detected by each tactile sensor is associated with a distribution of a plurality of tactile sensors arranged in the body as the tactile data (at least as in paragraph 0040, wherein “the sensor data may include images (e.g., including video) taken from cameras located on the robot or around the robot (e.g., with the robot or a workpiece in a field of view of the cameras)”; at least as in paragraph 0123, wherein “an array of cameras, like a stereoscopic pair (or set of 3, 5, 7 or more) of cameras may each have an associate hardware machine-learning accelerator that performs, for example, one or more convolutional layers or one or more pooling layers therebetween in a neural network and output of these two hardware machine-learning accelerators may then be merged into yet another hardware machine-learning accelerator that infers depth related features or vectors in an embedding spaces that encode information about depth”; at least as in paragraph 0069, wherein “an encoder 250B that receives input from a touch sensor matrix”; at least as in paragraph 0072, wherein “a sensor layer 240 of a robot system may provide both servo position data and tactile information, like in the form of pressure or contact readings from fingertip sensors, like in a matrix of force readings corresponding to a grid of sensors on each of a plurality of different end effectors”; at least as in paragraph 0077, wherein “ML accelerator 250B may execute an encoder model that ingests the sensor data output by a sensor 240B, like an array of touch sensors which reports values, like readouts of strings of touch values, such by row/column corresponding a plurality of touch sensors within the array. The encoder model may be a geometric learning model that outputs a vector indicative of size of touch area, force, and location within the array, like a vector within a latent embedding space which may distinguish between different areas, forces, and locations”), and
wherein the processor generates the first compressed data by fusing and dimensionally compressing the distribution of the plurality of tactile sensors and the depth image data (at least as in paragraph 0038, 0041, and 0069, wherein an encoder may dimensionally reduce the sensor data from vision sensors and tactile sensors into a lower-dimensional vector representation; therefore, Kranski teaches wherein the encoder dimensionally reduces depth information from camera images depicting the robot and workpiece and touch values from an array of touch sensors on the robot into a lower-dimensional vector representation).
Regarding claim 3, in view of the above combination of Kranski and Tee, Kranski further discloses the autonomous control system according to claim 2, wherein the processor generates the second compressed data by dimensionally compressing the data in which the contact force detected by each tactile sensor is associated with the distribution of the plurality of tactile sensors (at least as in paragraph 0069, wherein “an encoder 250B that receives input from a touch sensor matrix”; at least as in paragraph 0072, wherein “a sensor layer 240 of a robot system may provide both servo position data and tactile information, like in the form of pressure or contact readings from fingertip sensors, like in a matrix of force readings corresponding to a grid of sensors on each of a plurality of different end effectors”; at least as in paragraph 0077, wherein “ML accelerator 250B may execute an encoder model that ingests the sensor data output by a sensor 240B, like an array of touch sensors which reports values, like readouts of strings of touch values, such by row/column corresponding a plurality of touch sensors within the array. The encoder model may be a geometric learning model that outputs a vector indicative of size of touch area, force, and location within the array, like a vector within a latent embedding space which may distinguish between different areas, forces, and locations”; .
Regarding claim 4, in view of the above combination of Kranski and Tee, Kranski further discloses the autonomous control system according to claim 1,
wherein the first encoder is a neural network trained on the basis of a training dataset in which a state of a correct answer of the target of the task is labeled for the visual data and the tactile data (at least as in paragraph 0017, wherein “a robot control model (or models) may pipeline an encoder model and a learning model that may be trained with end-to-end learning … An encoder model may be operative to transform high-dimensional outputs of a robot's sensor suite into lower-dimensional vector representations of a slice in time … A learning model may be configured to update setpoints for robot actuators based on those vectors (e.g., based on their latent space embedding)”; at least as in paragraph 0076, wherein “the ML accelerator 250 may execute a convolutional neural network or a vision transformer to output a vector indicative of a slice of input data received from sensor 240A and 240B with in the latent space”; at least as in paragraph 0118, wherein “the hardware machine-learning accelerators may execute a trained deep neural network implementing an autoencoder that transforms relatively high dimensional data, like video and other sensor data (e.g., motor currents, position encoders, depth images, 3 or 6 axis IMU readings, tactile sensor outputs, or the like) into a lower dimensional representation, like a vector in an embedding space”; at least as in paragraph 0126, wherein “the high-dimensional data may be reduced by one or more encoder models (which each may implement a neural network) that process sensor data”; at least as in paragraph 0102, wherein “a machine learning model 302 may include a neural network or other machine learning model described herein, may take inputs 304 (e.g., input data that described above) and provide outputs 306 (e.g., output data like that described above) based on the inputs and parameter values of the model … outputs 306 may be fed back to machine learning model 302 as input to train machine learning model 302 (e.g., alone or in conjunction with indications of the performance of outputs 306, thresholds associated with the inputs, or with other feedback information) … machine learning model 302 may update its configurations (e.g., weights, biases, or other parameters) based on its assessment of a prediction or instructions (e.g., outputs 306) against feedback information (e.g., sensor data, state labels, like anomalous, indications of the performance or with other feedback information)”).
Regarding claim 7, in view of the above combination of Kranski and Tee, Kranski further discloses the autonomous control system according to claim 1, wherein the processor decides on the action from the combined state data using reinforcement learning (at least as in paragraph 0017, wherein “a reinforcement learning model that controls the robot (e.g., outputs a time series of target setpoints of a plurality of actuators) based on a time-series of those vectors, each vector representing a time-slice or robot and environment state”; at least as in paragraph 0027, wherein “The ML models may include an encoder model, a reinforcement learning model, a computer vision model, a geometric deep learning model, a dynamic model, an actor-critic model, a reward model, an anomaly detection model, or a variety of other machine learning models”; at least as in paragraph 0037, wherein “a robot control model 116 may include a reinforcement learning model trained at least in part via a reinforcement learning process, and the reinforcement learning model may take, as input, outputs of one or more encoder models”; at least as in paragraph 0068-0070, 0079, and 0044, wherein the encoder may combine the robot state data/information, the lower-dimensional vector of vision and tactile sensor data, and the dimensionally reduced touch sensor data).
Regarding claim 10, Kranski discloses an autonomous control method comprising:
acquiring state data of a robot, visual data of the robot, and tactile data of the robot (at least as in paragraph 0034, wherein “Sensors of a robot system 102 may output sensor data”; at least as in paragraph 0043, wherein “Various sensors, such as tactile or force sensors or strain sensors or pressure sensors, along with cameras, inertial measurement units, and the like may output sensor data corresponding to movements and interactions of components of the arm with itself or other objects. Sensor data may be collected from various image or distance sensors, which may be processed, such as by encoder models, to output vectors indicative of position of the arm (or members thereof) within the environment and other environmental data. For example, as the robot moves towards an object, sensor data including images showing the environment around the robot, data indicating positions of legs and arms of the robot, etc.”; at least as in paragraph 0044, wherein “Sensor data may include position data from servo motors or stepper motors indicating the reported positions of one or more part of the robot, the reported positions of one or more parts of the robot relative to other parts of the robot, battery level, power consumption, motor current, or a variety of other information associated with state of the robot. Sensor data may include information obtained from a motor position sensor of the robot (e.g., located in arm, member, joint, or other part of the robot system 102), a touch sensor located in a part of the robot system 102 (e.g., a finger of the robot system 102), or a motor current sensor of the robot”);
deciding on an action of the robot capable of accomplishing a task given to the robot on the basis of the state data, the visual data, and the tactile data (at least as in paragraph 0030, wherein “A trained control model 116A of the robot system 102A thus may account for (e.g., learn to accommodate) properties of the robot system 102A for which it generates instructions to perform robot control actions based on robot state, such as to cause the robot to complete a task”; at least as in paragraph 0102, wherein “the model 302 may be fed an input or set of inputs 304 for processing based on a state, sensor data, action, instructions for an action, or other data, and provide an output or set of outputs 306”);
generating first compressed data having a smaller number of dimensions than data (at least as in paragraph 0038, wherein “Each (or at least some, such as upstream encoders) of the encoder models may transform relatively high-dimensional outputs of a robot's sensor suite into lower-dimensional vector representations”; at least as in paragraph 0041, wherein “the plurality of channels of sensor data may be transformed into embedding vectors within different sub-spaces of the latent embedding space by a first set of encoder models coupled to the sensors”; at least as in paragraph 0067, wherein “sensor layer 240 may thus include a plurality of sensors 240A-E, which may include one or more computer vision sensors (e.g., various cameras, LiDAR, etc.), proximity sensors (e.g., ultrasonic, etc.), tactile or force sensors or strain sensors or pressure sensors, inertial measurement units, and the like, among other sources of feedback data, like servos, stepper motors, actuators and the like”; at least as in paragraph 0069, wherein “encoder 250F may ingest sensor feedback data from upstream encoder models (e.g., 250D, 250E) as shown, or from sensors (e.g., 240A-E) of the sensor layer 240. Each encoder 250 may perform dimensionality reduction on inputs, but the amount of reduction may vary, such as whether an encoder is performing reduction on encoder outputs, sensor outputs, a combination thereof, and the type of data”; at least as in paragraph 0070, wherein “an encoder 250A may receive inputs from two or more sensors 240A, 240B”; at least as in paragraph 0073, wherein “one or more layers of encoder models may be implemented by hardware machine-learning accelerators”; at least as in paragraph 0076, wherein “ML Accelerator 250A may ingest sensor data from sensors 240A and 240B … execute an encoder model that generates a latent-space embedding based on the combined sensor data 240A and 240B … execute a convolutional neural network or a vision transformer to output a vector indicative of a slice of input data received from sensor 240A and 240B with in the latent space. The output by the ML accelerator 250A may thus be of a lower dimensionality”; at least as in paragraph 0135, wherein “the first accelerator may obtain first sensor data from a first subset of sensors, like two or more sensors of the robot [and] transform inputs received via outputs of the first subset of sensors into a first sub-space representation that accounts for properties sensed by the first subset of sensors of the robot”; therefore, Kranski teaches wherein an encoder dimensionally reduces and combines data from a vision sensor and a tactile sensor into a lower-dimensional vector);
generating second compressed data having a smaller number of dimensions than the tactile data by dimensionally compressing the tactile data using a second encoder (at least as in paragraph 0069, wherein “an encoder 250B that receives input from a touch sensor matrix may reduce dimensionality of received inputs by 10× or 100”; therefore, Kranski teaches wherein an encoder dimensionally reduces touch sensor data);
deciding on the action on the basis of combined state data obtained by combining the state data, the first compressed data, and the second compressed data into one using a policy network (at least as in paragraph 0068, wherein “downstream encoder model 250F need not ingest each channel of sensor data directly, but rather may ingest representations of the sensor data from those channels that are output by upstream encoders”; at least as in paragraph 0069, wherein “Each encoder 250 may perform dimensionality reduction on inputs, but the amount of reduction may vary, such as whether an encoder is performing reduction on encoder outputs, sensor outputs, a combination thereof, and the type of data”; at least as in paragraph 0070, wherein “an encoder 250E may receive inputs from one or more encoders 250C (which receives inputs from one or more sensors 240D) and one or more sensors 240E”; at least as in paragraph 0079, wherein “a first accelerator may encode depth/distance information (e.g., from a LiDAR sensor), a second accelerator may encode object localization/detection data (e.g., from a camera), and a third encoder may combine the depth/distance information with the object localization/detection data without encountering the sensor information directly”; at least as in paragraph 0044, wherein “Sensor data may include position data from servo motors or stepper motors indicating the reported positions of one or more part of the robot, the reported positions of one or more parts of the robot relative to other parts of the robot, battery level, power consumption, motor current, or a variety of other information associated with state of the robot”; therefore, the encoder may combine the robot state data/information, the lower-dimensional vector of vision and tactile sensor data, and the dimensionally reduced touch sensor data; at least as in paragraph 0030, wherein “A trained control model 116A of the robot system 102A thus may account for (e.g., learn to accommodate) properties of the robot system 102A for which it generates instructions to perform robot control actions based on robot state, such as to cause the robot to complete a task”; at least as in paragraph 0075, wherein “a downstream encoder, like a HW ML Layer-3 ML Accelerator 250F may take input from one or more intermediate (e.g., Layer-2) accelerators 250D-E prior to providing output to a CPU 260, which may execute one or more control models that may cause the robot to perform an action based on the input it receives from the ML accelerator 250F”);
inputting the combined state data into the policy network (at least as in paragraph 0141, “the process may control the robot via one or more control models executed by one or more processor based on the latent-space representation. For example, a robot control model may include a reinforcement learning model trained at least in part via a reinforcement learning process, and the reinforcement learning model may take, as input, outputs of one or more encoder models. The encoder models executed by one or more ML Accelerators may simplify the input parameter space of the reinforcement learning model, which, due to complexity may be executed on a general purposed central processing unit. Reduction of the number of input parameters, for example, may reduce latency of model execution over a stream of input data”); and
generating a command that controls an actuator of the robot based on an action variable indicating the action that is output by the policy network in response to input of the combined state data (at least as in paragraph 0138, “In a step 425, the process may include controlling the robot based on the space embeddings. In some examples, a processor may use the combination of sub-space embeddings to control the robot”; at least as in paragraph 0141, “the process may control the robot via one or more control models executed by one or more processor based on the latent-space representation”).
Kranski does not explicitly disclose obtained by combining the visual data and the tactile data by fusing and dimensionally compressing the visual data and the tactile data using a first encoder.
However, Tee, in the same field of endeavor of robot control using a combination of vision and tactile data, specifically discloses “obtained by combining the visual data and the tactile data by fusing and dimensionally compressing the visual data and the tactile data using a first encoder” (at least as in paragraph 0175, “The system 1400 comprises a first spiking neural network, SNN, encoder 1402 configured for encoding an event based output of a vision sensor 1404 into individual vision modality spiking representations with a first output size; a second SNN encoder 1406 configured for encoding an event based output of a tactile sensor 1408 into individual tactile modality spiking representations with a second output size; a combination layer 1410 configured for merging the vision modality spiking representations and the tactile modality spiking representations; and a task SNN 1412 configured to receive the merged vision modality spiking representations and tactile modality spiking representations and output vision-tactile modality spiking representations with a third output size for classification”; at least as in paragraph 0182, “At step 1506, the vision modality spiking representations and the tactile modality spiking representations are merged, using a combination layer. At step 1508, using a task SNN to receive the merged vision modality spiking representations and tactile modality spiking representations and to output vision-tactile modality spiking representations with a third output size for classification, using a task SNN to receive the concatenated vision modality spiking representations and tactile modality spiking representations”; at least as in paragraph 0182, “At step 1504, an event-based output of a tactile sensor is encoded, using a second SNN encoder, into individual tactile modality spiking representations with a second output size”).
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Kranski, to include Tee's teaching of a system encoding and merging visual and tactile data into a smaller output size, since Tee teaches wherein the system encoding and merging the data reduces computational costs and latency thus improving the effectiveness of a reasonable hardware setup.
Regarding claim 11, Kranski discloses a computer-readable non-transitory storage medium storing a program (at least as in paragraph 0154, wherein “System memory 1020 may include a non-transitory computer readable storage medium that may have program instructions stored thereon that are executable by a computer processor (e.g., one or more of processors 1010a-1010n) to cause the subject matter and the functional operations described herein”) for causing a computer to:
acquire state data of a robot, visual data of the robot, and tactile data of the robot (at least as in paragraph 0034, wherein “Sensors of a robot system 102 may output sensor data”; at least as in paragraph 0043, wherein “Various sensors, such as tactile or force sensors or strain sensors or pressure sensors, along with cameras, inertial measurement units, and the like may output sensor data corresponding to movements and interactions of components of the arm with itself or other objects. Sensor data may be collected from various image or distance sensors, which may be processed, such as by encoder models, to output vectors indicative of position of the arm (or members thereof) within the environment and other environmental data. For example, as the robot moves towards an object, sensor data including images showing the environment around the robot, data indicating positions of legs and arms of the robot, etc.”; at least as in paragraph 0044, wherein “Sensor data may include position data from servo motors or stepper motors indicating the reported positions of one or more part of the robot, the reported positions of one or more parts of the robot relative to other parts of the robot, battery level, power consumption, motor current, or a variety of other information associated with state of the robot. Sensor data may include information obtained from a motor position sensor of the robot (e.g., located in arm, member, joint, or other part of the robot system 102), a touch sensor located in a part of the robot system 102 (e.g., a finger of the robot system 102), or a motor current sensor of the robot”);
decide on an action of the robot capable of accomplishing a task given to the robot on the basis of the state data, the visual data, and the tactile data (at least as in paragraph 0030, wherein “A trained control model 116A of the robot system 102A thus may account for (e.g., learn to accommodate) properties of the robot system 102A for which it generates instructions to perform robot control actions based on robot state, such as to cause the robot to complete a task”; at least as in paragraph 0102, wherein “the model 302 may be fed an input or set of inputs 304 for processing based on a state, sensor data, action, instructions for an action, or other data, and provide an output or set of outputs 306”);
generate first compressed data having a smaller number of dimensions than data obtained by combining the visual data and the tactile data by fusing and dimensionally compressing the visual data and the tactile data using a first encoder (at least as in paragraph 0038, wherein “Each (or at least some, such as upstream encoders) of the encoder models may transform relatively high-dimensional outputs of a robot's sensor suite into lower-dimensional vector representations”; at least as in paragraph 0041, wherein “the plurality of channels of sensor data may be transformed into embedding vectors within different sub-spaces of the latent embedding space by a first set of encoder models coupled to the sensors”; at least as in paragraph 0067, wherein “sensor layer 240 may thus include a plurality of sensors 240A-E, which may include one or more computer vision sensors (e.g., various cameras, LiDAR, etc.), proximity sensors (e.g., ultrasonic, etc.), tactile or force sensors or strain sensors or pressure sensors, inertial measurement units, and the like, among other sources of feedback data, like servos, stepper motors, actuators and the like”; at least as in paragraph 0069, wherein “encoder 250F may ingest sensor feedback data from upstream encoder models (e.g., 250D, 250E) as shown, or from sensors (e.g., 240A-E) of the sensor layer 240. Each encoder 250 may perform dimensionality reduction on inputs, but the amount of reduction may vary, such as whether an encoder is performing reduction on encoder outputs, sensor outputs, a combination thereof, and the type of data”; at least as in paragraph 0070, wherein “an encoder 250A may receive inputs from two or more sensors 240A, 240B”; at least as in paragraph 0073, wherein “one or more layers of encoder models may be implemented by hardware machine-learning accelerators”; at least as in paragraph 0076, wherein “ML Accelerator 250A may ingest sensor data from sensors 240A and 240B … execute an encoder model that generates a latent-space embedding based on the combined sensor data 240A and 240B … execute a convolutional neural network or a vision transformer to output a vector indicative of a slice of input data received from sensor 240A and 240B with in the latent space. The output by the ML accelerator 250A may thus be of a lower dimensionality”; at least as in paragraph 0135, wherein “the first accelerator may obtain first sensor data from a first subset of sensors, like two or more sensors of the robot [and] transform inputs received via outputs of the first subset of sensors into a first sub-space representation that accounts for properties sensed by the first subset of sensors of the robot”; therefore, Kranski teaches wherein an encoder dimensionally reduces and combines data from a vision sensor and a tactile sensor into a lower-dimensional vector);
generate second compressed data having a smaller number of dimensions than the tactile data by dimensionally compressing the tactile data using a second encoder (at least as in paragraph 0069, wherein “an encoder 250B that receives input from a touch sensor matrix may reduce dimensionality of received inputs by 10× or 100”; therefore, Kranski teaches wherein an encoder dimensionally reduces touch sensor data); and
decide on the action on the basis of combined state data obtained by combining the state data, the first compressed data, and the second compressed data into one using a policy network (at least as in paragraph 0068, wherein “downstream encoder model 250F need not ingest each channel of sensor data directly, but rather may ingest representations of the sensor data from those channels that are output by upstream encoders”; at least as in paragraph 0069, wherein “Each encoder 250 may perform dimensionality reduction on inputs, but the amount of reduction may vary, such as whether an encoder is performing reduction on encoder outputs, sensor outputs, a combination thereof, and the type of data”; at least as in paragraph 0070, wherein “an encoder 250E may receive inputs from one or more encoders 250C (which receives inputs from one or more sensors 240D) and one or more sensors 240E”; at least as in paragraph 0079, wherein “a first accelerator may encode depth/distance information (e.g., from a LiDAR sensor), a second accelerator may encode object localization/detection data (e.g., from a camera), and a third encoder may combine the depth/distance information with the object localization/detection data without encountering the sensor information directly”; at least as in paragraph 0044, wherein “Sensor data may include position data from servo motors or stepper motors indicating the reported positions of one or more part of the robot, the reported positions of one or more parts of the robot relative to other parts of the robot, battery level, power consumption, motor current, or a variety of other information associated with state of the robot”; therefore, the encoder may combine the robot state data/information, the lower-dimensional vector of vision and tactile sensor data, and the dimensionally reduced touch sensor data; at least as in paragraph 0030, wherein “A trained control model 116A of the robot system 102A thus may account for (e.g., learn to accommodate) properties of the robot system 102A for which it generates instructions to perform robot control actions based on robot state, such as to cause the robot to complete a task”; at least as in paragraph 0075, wherein “a downstream encoder, like a HW ML Layer-3 ML Accelerator 250F may take input from one or more intermediate (e.g., Layer-2) accelerators 250D-E prior to providing output to a CPU 260, which may execute one or more control models that may cause the robot to perform an action based on the input it receives from the ML accelerator 250F”);
input the combined state data into the policy network (at least as in paragraph 0141, “the process may control the robot via one or more control models executed by one or more processor based on the latent-space representation. For example, a robot control model may include a reinforcement learning model trained at least in part via a reinforcement learning process, and the reinforcement learning model may take, as input, outputs of one or more encoder models. The encoder models executed by one or more ML Accelerators may simplify the input parameter space of the reinforcement learning model, which, due to complexity may be executed on a general purposed central processing unit. Reduction of the number of input parameters, for example, may reduce latency of model execution over a stream of input data”); and
generate a command that controls an actuator of the robot based on an action variable indicating the action that is output by the policy network in response to input of the combined state data (at least as in paragraph 0138, “In a step 425, the process may include controlling the robot based on the space embeddings. In some examples, a processor may use the combination of sub-space embeddings to control the robot”; at least as in paragraph 0141, “the process may control the robot via one or more control models executed by one or more processor based on the latent-space representation”).
Kranski does not explicitly disclose obtained by combining the visual data and the tactile data by fusing and dimensionally compressing the visual data and the tactile data using a first encoder.
However, Tee, in the same field of endeavor of robot control using a combination of vision and tactile data, specifically discloses “obtained by combining the visual data and the tactile data by fusing and dimensionally compressing the visual data and the tactile data using a first encoder” (at least as in paragraph 0175, “The system 1400 comprises a first spiking neural network, SNN, encoder 1402 configured for encoding an event based output of a vision sensor 1404 into individual vision modality spiking representations with a first output size; a second SNN encoder 1406 configured for encoding an event based output of a tactile sensor 1408 into individual tactile modality spiking representations with a second output size; a combination layer 1410 configured for merging the vision modality spiking representations and the tactile modality spiking representations; and a task SNN 1412 configured to receive the merged vision modality spiking representations and tactile modality spiking representations and output vision-tactile modality spiking representations with a third output size for classification”; at least as in paragraph 0182, “At step 1506, the vision modality spiking representations and the tactile modality spiking representations are merged, using a combination layer. At step 1508, using a task SNN to receive the merged vision modality spiking representations and tactile modality spiking representations and to output vision-tactile modality spiking representations with a third output size for classification, using a task SNN to receive the concatenated vision modality spiking representations and tactile modality spiking representations”; at least as in paragraph 0182, “At step 1504, an event-based output of a tactile sensor is encoded, using a second SNN encoder, into individual tactile modality spiking representations with a second output size”).
Therefore, it would have been obvious to one of the ordinary skill in the art at the effective filing date of the instant invention to modify the teachings of Kranski, to include Tee's teaching of a system encoding and merging visual and tactile data into a smaller output size, since Tee teaches wherein the system encoding and merging the data reduces computational costs and latency thus improving the effectiveness of a reasonable hardware setup.
Claim(s) 5 and 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kranski et al. (US 20220314434 A1, hereinafter Kranski) in view of Tee et al. (US 20230330859 A1, hereinafter Tee), and further in view of Tachikake (US 20220234196 A1).
Regarding claim 5, in view of the above combination of Kranski and Tee, Kranski further discloses the autonomous control system according to claim 1,
wherein the first encoder is a neural network that converts input data into data having a smaller number of dimensions and outputs the data having the smaller number of dimensions and that is trained so that data input to the first encoder matches data output by a decoder (at least as in paragraph 0073, wherein “one or more layers of encoder models may be implemented by hardware machine-learning accelerators”; at least as in paragraph 0076, wherein “the ML accelerator 250 may execute a convolutional neural network or a vision transformer to output a vector indicative of a slice of input data received from sensor 240A and 240B with in the latent space. The output by the ML accelerator 250A may thus be of a lower dimensionality than the output of the vision sensor 240A and the proximity sensor 250A”; at least as in paragraph 0102, wherein “one or more neurons (or nodes) of the neural network may require that their respective errors are sent backward through the neural network to them to facilitate the update process (e.g., backpropagation of error)”; at least as in paragraph 0100, wherein “the data stored by the experience buffers 226 may be used by the actor-critic trainer to train the actor-critic model 206 to determine actions for the robot 216 to perform. Some models may be trained based on the outputs of other models. For example, the actor-critic model 206 and the encoder model 203 may be trained based on outputs generated by each other or other models. For example, a trainer may adjust a given weight of the encoder model 203 based on an action determined by a reinforcement learning model (e.g., the actor-critic model 206)”).
However, Kranski does not specifically disclose “in combination with the decoder that converts the input data into data having a larger number of dimensions and outputs the data having the larger number of dimensions.”
Tachikake discloses a machine learning data generation device includes a virtual model generator which generates virtual subject models of a plurality of randomly piled subjects to be subjected to physical work by an operating machine of a work machine. Tachikake specifical