Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations, under their broadest reasonable interpretation, cover mental process (concept performed in a human mind, including as observation, evaluation, judgment, opinion, organizing human activity and solving mathematical function and calculations). This judicial exception is not integrated into a practical application because the steps do not add meaningful limitations to be considered specifically applied to a particular technological problem to be solved .The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be done mentally using paper/pencil, solving mathematical function/problem and labelling the input and learning the model depending on the label. Further no additional features in the claims would preclude them from being performed as such except for the generic computer elements and generic model learning at high level of generality (i.e., processor, memory) .
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claims 1, 8 and 12 are directed to an abstract idea as shown below:
Regarding claims 1, 8 and 12
STEP 1: Do the claims fall within one of the statutory categories?
YES.
Claim(s) 1, 8 and 12 are directed to a method, a device and a non-transitory readable storage medium storing computer program, i.e. process, system and manufacture.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?
YES.
The claims are directed toward a mental process and solving mathematical problem (i.e. abstract idea).
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
claims 1, 8 and 12 comprise a mental process that can be practicably performed in the human mind and solving mathematical function/problem (or generic computers or components configured to perform the process learning the model i.e., machine learning and, therefore, an abstract idea.
Regarding Claims 1, 8 and 12 (representative claim 8):
A device for active learning from multimodal input, characterized in that the device comprises:
at least one processor and at least one memory that is configured to store instructions that when executed by the at least one processor (generic computing elements/components ) cause the device to:
providing the input and learning a model depending on the input, wherein the input comprises input of different modes (providing the collected data and mathematical function i.e., model which is based on the input and input include different modes which is insignificant extra solution activity and input includes different modes is mental process based on the collected data), and
learning the model comprises determining an input of a mode from the input of different modes depending on an acquisition function that comprises a measure for a cost for labelling the input of the mode (based on the human intelligence learning the mathematical function, which includes determining the input of a mode from input of different modes from the collected data depending on acquiring function which measure cost of labelling of label the data i.e. solving the mathematical function to label the data based on human intelligence),
labelling the input of the mode with a label (labelling the data of the mode i.e., mental process using paper/pencil) , and
learning the model depending on the label (learning the model depending on the label i.e., based on human intelligence learning solve mathematical function on paper/pencil based on label).
The above limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind or by a human intelligence and solving mathematical function/problem. Furthermore limitations, “at least one processor and at least one memory that is configured to store instructions that when executed by the at least one processor (generic computing elements/components ) cause the device to: providing the input and learning a model depending on the input, wherein the input comprises input of different modes (providing the collected data and mathematical function i.e., model which is based on the input and input include different modes which is insignificant extra solution activity and input includes different modes is mental process) and
learning the model comprises determining an input of a mode from the input of different modes depending on an acquisition function that comprises a measure for a cost for labelling the input of the mode (based on the human intelligence learning the mathematical function, which includes determining the input of a mode from input of different modes from the collected data depending on acquiring function which measure cost of labelling of label the data i.e. learning to solve the mathematical function to label the data based on human intelligence), labelling the input of the mode with a label (labelling the data of the mode i.e., mental process using paper/pencil) and learning the model depending on the label (learning the model depending on the label i.e., based on human intelligence learning to solve mathematical function on paper/pencil based on label)” are insignificant.
The Examiner notes that under MPEP 2106.04(A) (2) (III), the courts consider a mental process (thinking, human intelligence) that can be performed in the mind/intelligence using a paper and pencil to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[Mental processes and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978).
Furthermore the Examiner also notes that even if you combined the math with the mental process, a combination of abstract ideas don't make a claim eligible. See MPEP 2106.04(II)(A)(2): Because a judicial exception is not eligible subject matter, Bilski, 561 U.S. at 601, 95 USPQ2d at 1005-06 (quoting Chakrabarty, 447 U.S. at 309, 206 USPQ at 197 (1980)), if there are no additional claim elements besides the judicial exception, or if the additional claim elements merely recite another judicial exception, that is insufficient to integrate the judicial exception into a practical application. See, e.g., RecogniCorp, LLC v. Nintendo Co., 855 F.3d 1322, 1327, 122 USPQ2d 1377 (Fed. Cir. 2017) ("Adding one abstract idea (math) to another abstract idea (encoding and decoding) does not render the claim non-abstract").
Other than generic and well-known computer hardware/software recited in the independent claims 1, 8 and 12 disclosed in the specification, nothing in the claims 1, 8 and 12 elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process using paper/pencil and learning to solve mathematical function/model. Learning the model comprises determining an input of a mode from the input of different modes depending on an acquisition function that comprises a measure for a cost for labelling the input of the mode, labelling the input of the mode with a label and learning the model depending on the label i.e., machine learning recited in independent claims 1, 8 and 12 is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing hardware/software and learning the model are recited as just to automate the mental process of mathematical problem solving(Step 2A, prong 1 Test Abstract idea = Yes).
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
[YES/NO].
The claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim(s) 1, 8 and 12 do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application.
Claim(s) 1, 8 and 12 recite A device for active learning from multimodal input, characterized in that the device comprises (representative claim 1):
at least one processor and at least one memory that is configured to store instructions that when executed by the at least one processor (generic computing elements/components ) cause the device to:
providing the input and learning a model depending on the input, wherein the input comprises input of different modes (providing the collected data and mathematical function i.e., model which is based on the input and input include different modes which is insignificant extra solution activity and input includes different modes is mental process), and
learning the model comprises determining an input of a mode from the input of different modes depending on an acquisition function that comprises a measure for a cost for labelling the input of the mode (based on the human intelligence learning the mathematical function, which includes determining the input of a mode from input of different modes from the collected data depending on acquiring function which measure cost of labelling of label the data i.e. solving the mathematical function to label the data based on human intelligence),
labelling the input of the mode with a label (labelling the data of the mode i.e., mental process using paper/pencil) , and
learning the model depending on the label (learning the model depending on the label i.e., based on human intelligence learning to solve mathematical function on paper/pencil based on label).
These limitations are recited at a high level of generality (i.e. as a general action or calculation being taken based on the results of the acquiring step) and amounts to mere post solution actions, which is a form of insignificant extra-solution activity without further detail. Further, the claims 1, 8 and 12 are claimed generically and are operating in their ordinary capacity such that they do not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
As stated above, other than generic and well-known computer hardware/software recited in the independent claims 1, 8 and 12 disclosed in the specification, nothing in the claims 1, 8 and 12 elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process using paper/pencil and learning to solve mathematical function/model. Determining an input of a mode from the input of different modes depending on an acquisition function that comprises a measure for a cost for labelling the input of the mode, labelling the input of the mode with a label and learning the model depending on the label i.e., machine learning recited in independent claims 1, 8 and 12 is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing hardware/software and learning the model i.e. machine learning are recited as just to automate the mental process of mathematical problem solving (Step 2A, prong 2 Test Abstract idea = Yes).
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
NO.
The claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
As stated above, other than generic and well-known computer hardware/software recited in the independent claims 1, 8 and 12 disclosed in the specification, nothing in the claims 1, 8 and 12 elements preclude the processing from being performed as mental process, or merely based on the observations, evaluation, judgement, thought process using paper/pencil and learning to solve mathematical function/model. Determining an input of a mode from the input of different modes depending on an acquisition function that comprises a measure for a cost for labelling the input of the mode, labelling the input of the mode with a label and learning the model depending on the label i.e., machine learning recited in independent claims 1, 8 and 12 is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing hardware/software and learning the model i.e. machine learning are recited as just to automate the mental process of mathematical problem solving
Thus, since Claim(s) 1, 8 and 12 are: (a) directed toward an abstract idea and math problem solving, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, it is clear that Claim(s) 1, 8 and 15 are not eligible subject matter under 35 U.S.C 101 (Step 2B, Test Abstract idea = Yes).
Regarding dependent claims: 2-7 and 9-11 the additional limitations of claims 2-7 and 9-11 further limit the abstract idea of independent claims 1, 8 and 12. Claims 2-7 and 9-11 do not integrate the mental process into practical application or add significantly more to the mental process. Claims 2-7 and 9-11 further limit the abstract idea of independent claims 1, 8 and 12. The limitations of dependent claims 2-7 and 9-11 fall under (mental process including observation and evaluation, and judgement and mathematical function/problem solving which can be done mentally in the human mind) OR (insignificant pre/post-solution extra activity of generating/gathering data, performing mathematical calculation) OR (generic computers or components configured to perform the process), the generic machine learning/ learning model recited in the dependent claims 2-7 and 9-11 and as disclosed in the specification is a mere idea of a solution without details per MPEP 2106.05( f ) or the idea of a technological environment without detail per MPEP 2106.05 ( h ). The generic computing hardware/software and learning the model i.e. machine learning are recited as just to automate the mental process of mathematical problem solving
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-12 are rejected under 35 USC 103 as being unpatentable over RAVENTOS et al. (US 20220335258).
Regarding claims 1, 8 and 12 RAVENTOS disclose method/ device/ non-transitory medium storing thereon a computer program for active learning from multimodal input (RAVENTOS Figs. 2 and 4-5, paragraph 0004 disclose the present disclosure provide for systems and methods for managing multi-modal datasets in active learning pipelines and paragraph 0060 disclose By leveraging the previously, and properly trained model, inference may be run on the present dataset and the unlabeled data to automatically generate apply labels to such data. As described herein, the data from multiple sensors may be synchronized to infer labels for a multi-modal scene. As shown in block 512 [Fig. 5]. This corresponds to method/ device/ non-transitory medium storing thereon a computer program for active learning from multimodal input ) comprises:
at least one processor and at least one memory that is configured to store instructions that when executed by the at least one processor (RAVENTOS, Figs. 4-5 and paragraph 0006 ) cause the device/computer:
providing the input and learning a model depending on the input, wherein the input comprises input of different modes (RAVENTOS Figs. 2 and 4-5, paragraph 0004 disclose managing multi-modal datasets in active learning pipelines. Models that have been trained on datasets and have already been labeled may provide a good prior on the true labels for a multi-modal scene. Further aspects of the disclosure provide for combining inference results of several high-performing models, or ensembles, for a particular task improves the quality of these pseudo-labels . Aspects of the present disclosure may leverage properly versioned models (including model weights and which dataset they were trained on) to run inference and RAVENTOS disclose paragraph 0060 by leveraging the previously, and properly trained model, inference may be run on the present dataset and the unlabeled data to automatically generate apply labels to such data. As described herein, the data from multiple sensors may be synchronized to infer labels for a multi-modal scene. As shown in block 512 [Fig. 5] and paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information. The sensor outputs may be combined into a multi-modal frame, where a sequence of frames generates a video. Object detection models, such as a convolutional neural network (CNN), are trained to identify objects of interest in each frame. Each identified object may be labeled or annotated with a bounding box. For each frame of a sequence of frames, the model may output a classification label corresponding to each identified object and a bounding box corresponding to a location of each object. The object detection model may be referred to as the model. An annotated frame may refer to a frame that includes the generated bounding box. The annotated frame may also include the classification label or a reference to the classification label. All this obviously corresponds to providing the input and learning a model depending on the input, wherein the input comprises input of different modes), and
learning the model comprises determining an input of a mode from the input of different modes depending on an acquisition function that comprises a measure for a cost for labelling the input of the mode (RAVENTOS paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information. The sensor outputs may be combined into a multi-modal frame, where a sequence of frames generates a video. Object detection models, such as a convolutional neural network (CNN), are trained to identify objects of interest in each frame. Each identified object may be labeled or annotated with a bounding box. For each frame of a sequence of frames, the model may output a classification label corresponding to each identified object and a bounding box corresponding to a location of each object. The object detection model may be referred to as the model. An annotated frame may refer to a frame that includes the generated bounding box. The annotated frame may also include the classification label or a reference to the classification label and RAVENTOS Fig. 3 paragraph 0040 disclose the machine learning model 300 may output an inferred label (y) for one or more objects in each image (x). The inferred labels (y) may be received at a loss function 308 [cost]. The loss function 308 may compare the predicted labels (y) to the ground truth actions (y*). The prediction error is the difference (e.g., loss) between the predicted label (y) and the ground truth action (y*). The prediction error is output from the loss function 308 to the machine learning model 300. The error may be back-propagated through the machine learning model 500 to update the parameters. The training may be performed during an offline phase of the machine learning model 300. This obviously corresponds to learning the model comprises determining an input of a mode from the input of different modes depending on an acquisition function that comprises a measure for a cost for labelling the input of the mode ),
labelling the input of the mode with a label (RAVENTOS paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information. The sensor outputs may be combined into a multi-modal frame, where a sequence of frames generates a video. Object detection models, such as a convolutional neural network (CNN), are trained to identify objects of interest in each frame. Each identified object may be labeled or annotated with a bounding box and RAVENTOS Fig. 3 paragraph 0040 disclose the machine learning model 300 may output an inferred label (y) for one or more objects in each image (x). The inferred labels (y) may be received at a loss function 308 . The loss function 308 may compare the predicted labels (y) to the ground truth actions (y*). The prediction error is the difference (e.g., loss) between the predicted label (y) and the ground truth action (y*). The prediction error is output from the loss function 308 to the machine learning model 300. The error may be back-propagated through the machine learning model 500 to update the parameters. The training may be performed during an offline phase of the machine learning model 300. ) , and
learning the model depending on the label (RAVENTOS Fig. 3 paragraph 0040 disclose machine learning model 300 may output an inferred label (y) for one or more objects in each image (x). The inferred labels (y) may be received at a loss function 308 The loss function 308 may compare the predicted labels (y) to the ground truth actions (y*). The prediction error is the difference (e.g., loss) between the predicted label (y) and the ground truth action (y*). The prediction error is output from the loss function 308 to the machine learning model 300. The error may be back-propagated through the machine learning model 500 to update the parameters. This obviously corresponds to labelling the input of the mode with a label).
Therefore it would have been obvious to one of ordinary skill in the art, before the claimed invention was filed to provide the input and learning a model depending on the input, wherein the input comprises input of different modes, learn the model to determine an input of a mode from the input of different modes depending on an acquisition function that measure for a cost for label the input of the mode, learn the model depending on the label and label the input of the mode with a label as shown by RAVENTOS because such a process and system provides automated system for controlling the actions of autonomous driving based on the surrounding object detection and object classification stated in paragraph 0017-0018.
Regarding claim 2 RAVENTOS disclose the acquisition function comprises a measure of an uncertainty of the model with respect to the input of the mode (RAVENTOS paragraph 0035 disclose the inference module 212 of the present disclosure may use a deep learning architecture. The deep learning architecture may be embodied in a deep convolutional neural network (CNN). During training, the CNN may be presented with various viewpoints of various object categories. The network designer may want the CNN to output an estimate of an unknown object and corresponding pose with a high degree of confidence. Before training, the output produced by the CNN is likely to be incorrect, and so an error may be calculated between the actual output and the target output. The weights of the CNN may then be adjusted so that the output of the CNN is more closely aligned with the target e.g., ground truth. The error would obviously provide measure of certainty or uncertainty).
Regarding claim 3 RAVENTOS the method comprises active learning the model for different tasks, wherein the acquisition function comprises a measure for a synergy of learning the different tasks with the input of the mode (RAVENTOS Figs. 2 and 4-5, paragraph 0004 disclose managing multi-modal datasets in active learning pipelines. Models that have been trained on datasets and have already been labeled may provide a good prior on the true labels for a multi-modal scene. Further aspects of the disclosure provide for combining inference results of several high-performing models, or ensembles, for a particular task improves the quality of these pseudo-labels further. Aspects of the present disclosure may leverage properly versioned models (including model weights and which dataset they were trained on and paragraph 0033 disclose unique identifier of the inference model may be used to determine a Universally Unique Identifiers (UUID). The inference module 212 may be considered task-specific. As such, the inference module 212 may have one or more associated ontologies and it may be known beforehand what labels/annotation-type the inference module 212 is going to produce. This obviously corresponds to active learning the model for different tasks, wherein the acquisition function comprises a measure for a synergy of learning the different tasks with the input of the mode)
Regarding claim 4 RAVENTOS disclose determining inputs of different modes from the input of different modes depending on the acquisition function. (paragraph 0004 disclose managing multi-modal datasets in active learning pipelines. Models that have been trained on datasets and have already been labeled may provide a good prior on the true labels for a multi-modal scene. Further aspects of the disclosure provide for combining inference results of several high-performing models, or ensembles, for a particular task improves the quality of these pseudo-labels further. Aspects of the present disclosure may leverage properly versioned models (including model weights and which dataset they were trained on and paragraph 0033 disclose unique identifier of the inference model may be used to determine a Universally Unique Identifiers (UUID). The inference module 212 may be considered task-specific. As such, the inference module 212 may have one or more associated ontologies and it may be known beforehand what labels/annotation-type the inference module 212 is going to produce. This obviously corresponds determining inputs of different modes from the input of different modes depending on the acquisition function).
Regarding claim 5 RAVENTOS disclose providing the input comprises providing the input to comprise an input of the mode digital image, LiDAR image, radar image, ultrasound image, infrared image, or acoustic signal (RAVENTOS paragraph 0004 disclose paragraph 0004 disclose managing multi-modal datasets in active learning pipelines and paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information).
Regarding claim 6 RVENTOS disclose providing the input comprises capturing the input with sensors, in particular a digital image sensor, a LiDAR image sensor, a radar image sensor, an ultrasound image sensor, an infrared image sensor, or an acoustic signal sensor (RAVENTOS paragraph 0004 disclose paragraph 0004 disclose managing multi-modal datasets in active learning pipelines and paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information).
Regarding claim 7 RAVENTOS disclose learning the model to map the input with the model to an output, and wherein the method comprises actuating a technical system depending on the output (RAVENTOS Figs 2-4 paragraph 0004 disclose managing multi-modal datasets in active learning pipelines. Models that have been trained on datasets and have already been labeled may provide a good prior on the true labels for a multi-modal scene. Further aspects of the disclosure provide for combining inference results of several high-performing models, or ensembles, for a particular task improves the quality of these pseudo-labels further. Aspects of the present disclosure may leverage properly versioned models (including model weights and which dataset they were trained on. RAVENTOS PARAGRAPH 0017 disclose actions of autonomous agents and semi-autonomous agents may be controlled or adjusted based on objects detected within a vicinity of the agent. For example, a route may be planned for an autonomous agent based on the locations of other objects on the road. As another example, a route may be adjusted to avoid a collision if a detected object is in the path of the agent. In the present disclosure, an agent refers to an autonomous agent or a semi-autonomous agent. RAVENTOS paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information. The sensor outputs may be combined into a multi-modal frame, where a sequence of frames generates a video. Object detection models, such as a convolutional neural network (CNN), are trained to identify objects of interest in each frame. Each identified object may be labeled or annotated with a bounding box. This obviously corresponds to learning the model to map the input with the model to an output. All this obviously corresponds to learning the model to map the input with the model to an output, and wherein the method comprises actuating a technical system depending on the output).
Regarding claim 9 RAVENTOS disclose device comprises sensors or an interface for sensors for capturing the input (RAVENTOS paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information. The sensor outputs may be combined into a multi-modal frame, where a sequence of frames generates a video. Object detection models, such as a convolutional neural network (CNN), are trained to identify objects of interest in each frame. Each identified object may be labeled or annotated with a bounding box).
Regarding claim 10 RAVENTOS disclose device comprises an actuator or an interface for an actuator for actuating a technical system (RAVENTOS PARAGRAPH 0017 disclose actions of autonomous agents and semi-autonomous agents may be controlled or adjusted based on objects detected within a vicinity of the agent. For example, a route may be planned for an autonomous agent based on the locations of other objects on the road. As another example, a route may be adjusted to avoid a collision if a detected object is in the path of the agent. In the present disclosure, an agent refers to an autonomous agent or a semi-autonomous agent. RAVENTOS paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information. The sensor outputs may be combined into a multi-modal frame, where a sequence of frames generates a video. Object detection models, such as a convolutional neural network (CNN), are trained to identify objects of interest in each frame. Each identified object may be labeled or annotated with a bounding box.)
Regarding claim 11 RAVENTOS disclose the device is comprises a portion of a technical system (RAVENTOS Figs. 2-4 RAVENTOS PARAGRAPH 0017 disclose actions of autonomous agents and semi-autonomous agents may be controlled or adjusted based on objects detected within a vicinity of the agent. For example, a route may be planned for an autonomous agent based on the locations of other objects on the road. As another example, a route may be adjusted to avoid a collision if a detected object is in the path of the agent. In the present disclosure, an agent refers to an autonomous agent or a semi-autonomous agent. RAVENTOS paragraph 0018 disclose the agent may detect objects from outputs of various sensors. For example, a 2D camera may generate 2D red-green-blue (RGB) images and a light detection and ranging (LIDAR) sensor may generate a 3D point cloud that provides height and depth information. The sensor outputs may be combined into a multi-modal frame, where a sequence of frames generates a video. Object detection models, such as a convolutional neural network (CNN), are trained to identify objects of interest in each frame. Each identified object may be labeled or annotated with a bounding box).
Communication
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ISHRAT I SHERALI whose telephone number is (571)272-7398. The examiner can normally be reached Monday-Friday 8:00AM -5:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
ISHRAT I. SHERALI
Examiner
Art Unit 2667
/ISHRAT I SHERALI/Primary Examiner, Art Unit 2667
Prosecution Insights