DETAILED ACTION
Notice of Pre-AIA or AIA Status
Claims 1-10 are pending in this application. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 9 and 10 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Rao et al. (US PGPub US 2024/0118667 A1, hereby referred to as “Rao”).
Consider Claims 1, 9 and 10.
Rao teaches:
1. (Currently Amended) A trained model generating device comprising: / 9. (Currently Amended) A trained model generating method comprising: / 10. (Currently Amended) A recognition device comprising: (Rao: abstract, Implementations disclosed herein relate to mitigating the reality gap through training a simulation-to-real machine learning model (“Sim2Real” model) using a vision-based robot task machine learning model. The vision-based robot task machine learning model can be, for example, a reinforcement learning (“RL”) neural network model (RL-network), such as an RL-network that represents a Q-function. Figure 1 [0031]-[0041], [0035] All or aspects of training system 150 may be implemented by the robot 130 in some implementations. In some implementations, all or aspects of training system 150 may be implemented by one or more remote computing systems and/or devices that are in communication with the robot 130 (e.g., computing device 910 of FIG. 9 ) over network(s) (e.g., local area network (LAN), wide area network (WAN), Bluetooth, and/or other networks). Various modules or engines may be implemented as part of training system 150 as software, hardware, or any combination of the two. For example, as shown in FIG. 1 , training system 150 can include a robotic simulator 152, simulation-to-real (“Sim2Real”) engine(s) 154, real-to-simulation (“Real2Sim”) engine(s) 156, vision-based robot task engine(s) 158, and loss engine(s) 160. Figures 3A-B)
1. a controller configured to generate a trained model that outputs a recognition result of a recognition target contained in input information, wherein the controller is further configured to (Rao: [0053]-[0054], [0053] RL provides a powerful tool to automate acquisition of visual representations for end-to-end training directly on a task-specific objective (e.g., grasping, path planning, navigation, etc.). For example, RL can be used to train deep neural network models (e.g., stored in vision based task model(s) database 190) for robots (e.g., robot 130) to grasp objects directly with image observations, or perform navigation with a mobile robot (e.g., robot 130) directly from on-board sensor readings of the mobile robot. However, the ability to learn visual representations end-to-end together with a task controller is costly in terms of collecting training data, and since the data needed for RL is typically task and policy specific, collecting this training data can be onerous. Accordingly, an appealing alternative is to generate simulated training data in simulated environments using simulated visual representations, and then use policies generated based on this simulated training data in real-world systems. However, by using this simulated training data, the policies result in improved representations of simulated environments, which may not accurately reflect real-world environments visually or physically (e.g., reality gap). [0103] The robot control system 660 may be implemented in one or more processors, such as a CPU, GPU, and/or other controller(s) of the robot 620. In some implementations, the robot 620 may comprise a “brain box” that may include all or aspects of the control system 660. For example, the brain box may provide real time bursts of data to the operational components 204 a-n, with each of the real time bursts comprising a set of one or more control commands that dictate, inter alia, the parameters of motion (if any) for each of one or more of the operational components 604 a-n. In various implementations, the control commands can be at least selectively generated by the control system 660 based at least in part on selected robot actions and/or other determination(s) made using a machine learning model that is stored locally on the robot 620 and that is trained according to implementations described herein.)
1. acquire a base model which includes a first base model, and which is trained using first information, the first information being identical to or related to the input information; / 9. acquiring a base model which includes a first base model, and which is trained using first information, the first information being identical to or related to the input information / 10. a trained model that outputs a recognition result of a recognition target contained in input information (Rao: [0044], [0061] Moreover, and as noted above, a CycleGAN model employ two GAN models that each include a simulator model and a discriminator model for both Sim2Real and Real2Sim (e.g., stored in the Sim2Real and Real2Sim model(s) database 180). In some implementations, a first GAN generator model of a given CycleGAN model can be a Sim2Real generator model that the Sim2Real engine(s) 154 can utilize to adapt an image from a source domain (e.g., simulated environment) to a target domain (e.g., real-world environment). In some further versions of those implementations, a second GAN generator model of the given CycleGAN model can be a Real2Sim generator model that the Real2Sim engine(s) 156 can utilize to adapt an image from a target domain (e.g., real-world environment) to a source domain (e.g., simulated environment). Further, the loss engine(s) 160 can generate a cycle consistency loss that ensures that when a given image is applied the first GAN model and the second GAN model in succession, that the adapted image is the given image. This enable the given CycleGAN model to preserve as much of a scene during adaptation since the image is reproduced from the adapted image by the second GAN model. Accordingly, visual differences caused by the reality gap can be mitigated because semantics of the scene (e.g., objects, robot pose(s), and/or other semantics) are preserved. [0078], [0091] Turning now to FIG. 5 , a flowchart illustrating an example method 500 of training a Sim2Real model according to various implementations disclosed herein. For convenience, the operations of the flow chart are described with reference to a system that performs the operations. This system may include one or more processors, such as processor(s) of robot 130 of FIG. 1 and/or robot 620 of FIG. 6 , processor(s) 714 of computing device of FIG. 7 , and/or other processor(s). While operations of method 500 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added. [0092] At block 552, the system processes a simulated image, using a simulation-to-real generator model, to generate a simulated episode predicted real image. The simulated image can be generated by a robotic simulator during a simulated episode of a simulated robot attempting performance of a robotic task. [0093]-[0094] At block 556, the system processes the simulated image along with a simulated robot action, using a task machine learning mode trained for use in the robotic task, to generate a first predicted value. In some implementations, the first predicted value is generated by applying the simulated image along with a simulated robot action as input across a value-network. In some other implementations, the first predicted value is a first Q-value generated by applying the simulated image along with a simulated robot action as input across a Q-network.)
1. generate a target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second target model, the second information being identical to or related to the input information; / 9. generating a target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second model, the second information being identical to or related to the input information; / 10. and the trained model comprising a base model which includes first model and which is trained using first information, the first information being identical to or related to the input information (Examiner Note: The data created in the simulation-to-real (“Sim2Real”) engine(s) 154, real-to-simulation (“Real2Sim”) engine(s) 156 would be identical in nature; Rao: [0061] Moreover, and as noted above, a CycleGAN model employ two GAN models that each include a simulator model and a discriminator model for both Sim2Real and Real2Sim (e.g., stored in the Sim2Real and Real2Sim model(s) database 180). In some implementations, a first GAN generator model of a given CycleGAN model can be a Sim2Real generator model that the Sim2Real engine(s) 154 can utilize to adapt an image from a source domain (e.g., simulated environment) to a target domain (e.g., real-world environment). In some further versions of those implementations, a second GAN generator model of the given CycleGAN model can be a Real2Sim generator model that the Real2Sim engine(s) 156 can utilize to adapt an image from a target domain (e.g., real-world environment) to a source domain (e.g., simulated environment). Further, the loss engine(s) 160 can generate a cycle consistency loss that ensures that when a given image is applied the first GAN model and the second GAN model in succession, that the adapted image is the given image. This enable the given CycleGAN model to preserve as much of a scene during adaptation since the image is reproduced from the adapted image by the second GAN model. Accordingly, visual differences caused by the reality gap can be mitigated because semantics of the scene (e.g., objects, robot pose(s), and/or other semantics) are preserved. [0078] For example, as shown in FIG. 3B, vision-based robot task engine 332 can process the simulated image 311 along with a simulated robot action 399 to generate a first predicted value 361, process the simulated episode predicted real image 312 along with the simulated robot action 399 to generate a second predicted value 362, and process the simulated episode predicted simulated image 313 along with the simulated robot action 399 to generate a third predicted value 363. The simulated robot action 399 can be any vision-based action that can be performed by a simulated robot during a simulated robotic task (e.g., a simulated object manipulation task, a simulated navigation task, a simulated path planning task, and/or any other simulated vision-based action). Moreover, the vision-based robot task engine 332 can process each of the images 311, 312, and 313 of the Sim2Real2Sim image triple using vision-based robot task model(s) (e.g., stored in the vision-based robot task model(s) database 190 of FIG. 1 ).)
1. acquire an adapter trained using at least third information in a state where the adapter is coupled to the base model, the third information being identical to or related to the input information; / 9. acquiring an adapter trained using at least third information in a state where the adapter is coupled to the base model, the third information being identical to or related to the input information; / 10. target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second target model, the second information representing the recognition target; (Examiner Note: The data created in the simulation-to-real (“Sim2Real”) engine(s) 154, real-to-simulation (“Real2Sim”) engine(s) 156 would be identical in nature; Rao: [0078] For example, as shown in FIG. 3B, vision-based robot task engine 332 can process the simulated image 311 along with a simulated robot action 399 to generate a first predicted value 361, process the simulated episode predicted real image 312 along with the simulated robot action 399 to generate a second predicted value 362, and process the simulated episode predicted simulated image 313 along with the simulated robot action 399 to generate a third predicted value 363. [0080] Loss engine 340D can process the first predicted value 361, the second predicted value 362, and the third predicted value 363 to generate a Sim2Real2Sim RL-scene consistency loss 340D1. More particularly, the loss engine 340D can determine a mean-squared error based on the first predicted value 361, the second predicted value 362, and the third predicted value 363, and generate the Sim2Real2Sim RL-scene consistency loss 340D1 as a function of these errors. Generating the Sim2Real2Sim RL-scene consistency loss 340D1 is described in greater detail above (e.g., with respect to FIG. 1 and Equation 4). Further, and although not depicted in FIG. 3B, a temporal difference (“TD”) loss can be generated based on the processing by the vision-based robot task engine 332 as described in greater detail above (e.g., with respect to FIG. 1 and Equation 5). The Sim2Real2Sim RL-scene consistency loss 340D1 and/or the TD loss can be utilized in generating an update that is backpropagated across the Sim2Real2Sim CycleGAN model of FIG. 3B, thereby training the Sim2Real model during the simulated episode of the robotic task. [0095] At block 558, the system processes the simulated episode predicted real image along with the simulated robot action, using the task machine learning model, to generate a second predicted value. In some implementations, the second predicted value is generated by applying the simulated image along with a simulated robot action as input across a value-network. In some other implementations, the second predicted value is a second Q-value generated by applying the simulated image along with a simulated robot action as input across a Q-network.)
1. and generate the trained model by coupling the adapter to the target model. / 9. and generating the trained model by coupling the adapter to the target model. / 10. and an adapter trained using at least third information as in a state the adapter is coupled to the base model, the adapter being coupled to the target model. (Rao: [0096] At block 560, the system processes the simulated episode predicted simulation image along with the simulated robot action, using the task machine learning model, to generate a third predicted value. In some implementations, the third predicted value is generated by applying the simulated image along with a simulated robot action as input across a value-network. In some other implementations, the third predicted value is a third Q-value generated by applying the simulated image along with a simulated robot action as input across a Q-network. [0097] At block 562, the system generates a loss as a function of comparisons of the first predicted value, the second predicted value, and the third predicted value. These comparisons comprise three comparisons between unique pairs of the first predicted value, the second predicted value, and the third predicted value (e.g., [first predicted value, second predicted value], [first predicted value, third predicted value], and [second predicted value, third predicted value]). By comparing these unique pairs of predicted values, a variance across the first predicted value, the second predicted value, and the third predicted value can be determined. The loss generated at block 562 can be generated as a function of the variance. Again, because each image in the image triple of [simulated image, simulated episode predicted real image, simulated episode predicted simulation image] represents the same scene, ideally there is little to no variance in these predicted values.)
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the invention was made.
Claims 1-10 are rejected under 35 U.S.C. 103 as being unpatentable over A et al. (US PGPub US 2024/0118667 A1 filed on May 15, 2020 with priority to May 4, 2020), hereby referred to as “Rao”, in view of Xu et al. (US PGPub US 2023/0042234 A1, filed on Oct 24, 2022 with priority to Oct 26, 2021), hereby referred to as “Xu”.
Consider Claims 1, 9 and 10.
Rao teaches:
1. (Currently Amended) A trained model generating device comprising: / 9. (Currently Amended) A trained model generating method comprising: / 10. (Currently Amended) A recognition device comprising: (Rao: abstract, Implementations disclosed herein relate to mitigating the reality gap through training a simulation-to-real machine learning model (“Sim2Real” model) using a vision-based robot task machine learning model. The vision-based robot task machine learning model can be, for example, a reinforcement learning (“RL”) neural network model (RL-network), such as an RL-network that represents a Q-function. Figure 1 [0031]-[0041], [0035] All or aspects of training system 150 may be implemented by the robot 130 in some implementations. In some implementations, all or aspects of training system 150 may be implemented by one or more remote computing systems and/or devices that are in communication with the robot 130 (e.g., computing device 910 of FIG. 9 ) over network(s) (e.g., local area network (LAN), wide area network (WAN), Bluetooth, and/or other networks). Various modules or engines may be implemented as part of training system 150 as software, hardware, or any combination of the two. For example, as shown in FIG. 1 , training system 150 can include a robotic simulator 152, simulation-to-real (“Sim2Real”) engine(s) 154, real-to-simulation (“Real2Sim”) engine(s) 156, vision-based robot task engine(s) 158, and loss engine(s) 160. Figures 3A-B)
1. a controller configured to generate a trained model that outputs a recognition result of a recognition target contained in input information, wherein the controller is further configured to (Rao: [0053]-[0054], [0053] RL provides a powerful tool to automate acquisition of visual representations for end-to-end training directly on a task-specific objective (e.g., grasping, path planning, navigation, etc.). For example, RL can be used to train deep neural network models (e.g., stored in vision based task model(s) database 190) for robots (e.g., robot 130) to grasp objects directly with image observations, or perform navigation with a mobile robot (e.g., robot 130) directly from on-board sensor readings of the mobile robot. However, the ability to learn visual representations end-to-end together with a task controller is costly in terms of collecting training data, and since the data needed for RL is typically task and policy specific, collecting this training data can be onerous. Accordingly, an appealing alternative is to generate simulated training data in simulated environments using simulated visual representations, and then use policies generated based on this simulated training data in real-world systems. However, by using this simulated training data, the policies result in improved representations of simulated environments, which may not accurately reflect real-world environments visually or physically (e.g., reality gap). [0103] The robot control system 660 may be implemented in one or more processors, such as a CPU, GPU, and/or other controller(s) of the robot 620. In some implementations, the robot 620 may comprise a “brain box” that may include all or aspects of the control system 660. For example, the brain box may provide real time bursts of data to the operational components 204 a-n, with each of the real time bursts comprising a set of one or more control commands that dictate, inter alia, the parameters of motion (if any) for each of one or more of the operational components 604 a-n. In various implementations, the control commands can be at least selectively generated by the control system 660 based at least in part on selected robot actions and/or other determination(s) made using a machine learning model that is stored locally on the robot 620 and that is trained according to implementations described herein.)
1. acquire a base model which includes a first base model, and which is trained using first information, the first information being identical to or related to the input information; / 9. acquiring a base model which includes a first base model, and which is trained using first information, the first information being identical to or related to the input information / 10. a trained model that outputs a recognition result of a recognition target contained in input information (Examiner Note: The data created in the simulation-to-real (“Sim2Real”) engine(s) 154, real-to-simulation (“Real2Sim”) engine(s) 156 would be identical in nature; Rao: [0044], [0061] Moreover, and as noted above, a CycleGAN model employ two GAN models that each include a simulator model and a discriminator model for both Sim2Real and Real2Sim (e.g., stored in the Sim2Real and Real2Sim model(s) database 180). In some implementations, a first GAN generator model of a given CycleGAN model can be a Sim2Real generator model that the Sim2Real engine(s) 154 can utilize to adapt an image from a source domain (e.g., simulated environment) to a target domain (e.g., real-world environment). In some further versions of those implementations, a second GAN generator model of the given CycleGAN model can be a Real2Sim generator model that the Real2Sim engine(s) 156 can utilize to adapt an image from a target domain (e.g., real-world environment) to a source domain (e.g., simulated environment). Further, the loss engine(s) 160 can generate a cycle consistency loss that ensures that when a given image is applied the first GAN model and the second GAN model in succession, that the adapted image is the given image. This enable the given CycleGAN model to preserve as much of a scene during adaptation since the image is reproduced from the adapted image by the second GAN model. Accordingly, visual differences caused by the reality gap can be mitigated because semantics of the scene (e.g., objects, robot pose(s), and/or other semantics) are preserved. [0078], [0091] Turning now to FIG. 5 , a flowchart illustrating an example method 500 of training a Sim2Real model according to various implementations disclosed herein. For convenience, the operations of the flow chart are described with reference to a system that performs the operations. This system may include one or more processors, such as processor(s) of robot 130 of FIG. 1 and/or robot 620 of FIG. 6 , processor(s) 714 of computing device of FIG. 7 , and/or other processor(s). While operations of method 500 are shown in a particular order, this is not meant to be limiting. One or more operations may be reordered, omitted, or added. [0092] At block 552, the system processes a simulated image, using a simulation-to-real generator model, to generate a simulated episode predicted real image. The simulated image can be generated by a robotic simulator during a simulated episode of a simulated robot attempting performance of a robotic task. [0093]-[0094] At block 556, the system processes the simulated image along with a simulated robot action, using a task machine learning mode trained for use in the robotic task, to generate a first predicted value. In some implementations, the first predicted value is generated by applying the simulated image along with a simulated robot action as input across a value-network. In some other implementations, the first predicted value is a first Q-value generated by applying the simulated image along with a simulated robot action as input across a Q-network.)
1. generate a target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second target model, the second information being identical to or related to the input information; / 9. generating a target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second model, the second information being identical to or related to the input information; / 10. and the trained model comprising a base model which includes first model and which is trained using first information, the first information being identical to or related to the input information (Examiner Note: The data created in the simulation-to-real (“Sim2Real”) engine(s) 154, real-to-simulation (“Real2Sim”) engine(s) 156 would be identical in nature; Rao: [0061] Moreover, and as noted above, a CycleGAN model employ two GAN models that each include a simulator model and a discriminator model for both Sim2Real and Real2Sim (e.g., stored in the Sim2Real and Real2Sim model(s) database 180). In some implementations, a first GAN generator model of a given CycleGAN model can be a Sim2Real generator model that the Sim2Real engine(s) 154 can utilize to adapt an image from a source domain (e.g., simulated environment) to a target domain (e.g., real-world environment). In some further versions of those implementations, a second GAN generator model of the given CycleGAN model can be a Real2Sim generator model that the Real2Sim engine(s) 156 can utilize to adapt an image from a target domain (e.g., real-world environment) to a source domain (e.g., simulated environment). Further, the loss engine(s) 160 can generate a cycle consistency loss that ensures that when a given image is applied the first GAN model and the second GAN model in succession, that the adapted image is the given image. This enable the given CycleGAN model to preserve as much of a scene during adaptation since the image is reproduced from the adapted image by the second GAN model. Accordingly, visual differences caused by the reality gap can be mitigated because semantics of the scene (e.g., objects, robot pose(s), and/or other semantics) are preserved. [0078] For example, as shown in FIG. 3B, vision-based robot task engine 332 can process the simulated image 311 along with a simulated robot action 399 to generate a first predicted value 361, process the simulated episode predicted real image 312 along with the simulated robot action 399 to generate a second predicted value 362, and process the simulated episode predicted simulated image 313 along with the simulated robot action 399 to generate a third predicted value 363. The simulated robot action 399 can be any vision-based action that can be performed by a simulated robot during a simulated robotic task (e.g., a simulated object manipulation task, a simulated navigation task, a simulated path planning task, and/or any other simulated vision-based action). Moreover, the vision-based robot task engine 332 can process each of the images 311, 312, and 313 of the Sim2Real2Sim image triple using vision-based robot task model(s) (e.g., stored in the vision-based robot task model(s) database 190 of FIG. 1 ).)
1. acquire an adapter trained using at least third information in a state where the adapter is coupled to the base model, the third information being identical to or related to the input information; / 9. acquiring an adapter trained using at least third information in a state where the adapter is coupled to the base model, the third information being identical to or related to the input information; / 10. target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second target model, the second information representing the recognition target; (Rao: [0078] For example, as shown in FIG. 3B, vision-based robot task engine 332 can process the simulated image 311 along with a simulated robot action 399 to generate a first predicted value 361, process the simulated episode predicted real image 312 along with the simulated robot action 399 to generate a second predicted value 362, and process the simulated episode predicted simulated image 313 along with the simulated robot action 399 to generate a third predicted value 363. [0080] Loss engine 340D can process the first predicted value 361, the second predicted value 362, and the third predicted value 363 to generate a Sim2Real2Sim RL-scene consistency loss 340D1. More particularly, the loss engine 340D can determine a mean-squared error based on the first predicted value 361, the second predicted value 362, and the third predicted value 363, and generate the Sim2Real2Sim RL-scene consistency loss 340D1 as a function of these errors. Generating the Sim2Real2Sim RL-scene consistency loss 340D1 is described in greater detail above (e.g., with respect to FIG. 1 and Equation 4). Further, and although not depicted in FIG. 3B, a temporal difference (“TD”) loss can be generated based on the processing by the vision-based robot task engine 332 as described in greater detail above (e.g., with respect to FIG. 1 and Equation 5). The Sim2Real2Sim RL-scene consistency loss 340D1 and/or the TD loss can be utilized in generating an update that is backpropagated across the Sim2Real2Sim CycleGAN model of FIG. 3B, thereby training the Sim2Real model during the simulated episode of the robotic task. [0095] At block 558, the system processes the simulated episode predicted real image along with the simulated robot action, using the task machine learning model, to generate a second predicted value. In some implementations, the second predicted value is generated by applying the simulated image along with a simulated robot action as input across a value-network. In some other implementations, the second predicted value is a second Q-value generated by applying the simulated image along with a simulated robot action as input across a Q-network.)
1. and generate the trained model by coupling the adapter to the target model. / 9. and generating the trained model by coupling the adapter to the target model. / 10. and an adapter trained using at least third information as in a state the adapter is coupled to the base model, the adapter being coupled to the target model. (Rao: [0096] At block 560, the system processes the simulated episode predicted simulation image along with the simulated robot action, using the task machine learning model, to generate a third predicted value. In some implementations, the third predicted value is generated by applying the simulated image along with a simulated robot action as input across a value-network. In some other implementations, the third predicted value is a third Q-value generated by applying the simulated image along with a simulated robot action as input across a Q-network. [0097] At block 562, the system generates a loss as a function of comparisons of the first predicted value, the second predicted value, and the third predicted value. These comparisons comprise three comparisons between unique pairs of the first predicted value, the second predicted value, and the third predicted value (e.g., [first predicted value, second predicted value], [first predicted value, third predicted value], and [second predicted value, third predicted value]). By comparing these unique pairs of predicted values, a variance across the first predicted value, the second predicted value, and the third predicted value can be determined. The loss generated at block 562 can be generated as a function of the variance. Again, because each image in the image triple of [simulated image, simulated episode predicted real image, simulated episode predicted simulation image] represents the same scene, ideally there is little to no variance in these predicted values.)
Even if Rao does not specifically teach: the first information being identical to or related to the input information; and the second information being identical to or related to the input information
Xu teaches:
1. (Currently Amended) A trained model generating device comprising: / 9. (Currently Amended) A trained model generating method comprising: / 10. (Currently Amended) A recognition device comprising: (Xu: abstract, A method for training a model includes: obtaining a scene image, second actual characters in the scene image and a second construct image; obtaining first features and first recognition characters of characters obtained by performing character recognition on the scene image using the model to be trained; obtaining second features of characters obtained by performing character recognition on the second construct image using the training auxiliary model; and obtaining a character recognition model by adjusting model parameters of the model to be trained based on the first recognition characters, the second actual characters, the first features and the second features. [0262], Figure 14)
1. a controller configured to generate a trained model that outputs a recognition result of a recognition target contained in input information, wherein the controller is further configured to (Xu: [0005] According to a first aspect of the disclosure, a method for training a model is provided. The method includes: obtaining a model to be trained and a training auxiliary model by training an initial neural network model based on a first construct image and first actual characters in the first construct image; obtaining a scene image, second actual characters in the scene image and a second construct image, in which characters in the second construct image are identical to the second actual characters; obtaining first features and first recognition characters of characters obtained by performing character recognition on the scene image using the model to be trained; obtaining second features of characters obtained by performing character recognition on the second construct image using the training auxiliary model; and obtaining a character recognition model by adjusting model parameters of the model to be trained based on the first recognition characters, the second actual characters, the first features and the second features. [0263] As illustrated in FIG. 14 , the electronic device 1400 includes: a computing unit 1401 performing various appropriate actions and processes based on computer programs stored in a read-only memory (ROM) 1402 or computer programs loaded from the storage unit 1408 to a random access memory (RAM) 1403. In the RAM 1403, various programs and data required for the operation of the device 1400 are stored. The computing unit 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. An input/output (I/O) interface 1405 is also connected to the bus 1404. [0265] The computing unit 1401 may be various general-purpose and/or dedicated processing components with processing and computing capabilities. Some examples of computing unit 1401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated AI computing chips, various computing units that run machine learning model algorithms, and a digital signal processor (DSP), and any appropriate processor, controller and microcontroller.)
1. acquire a base model which includes a first base model, and which is trained using first information, the first information being identical to or related to the input information; / 9. acquiring a base model which includes a first base model, and which is trained using first information, the first information being identical to or related to the input information / 10. a trained model that outputs a recognition result of a recognition target contained in input information (Xu: [0029] As illustrated in FIG. 1 , FIG. 1 is a flowchart of a first method for training a model according to embodiments of the disclosure. The above method includes the following steps S101-S105. [0030] At block S101, a model to be trained and a training auxiliary model are obtained by training an initial neural network model based on a first construct image and first actual characters in the first construct image. [0031] The above first construct image refers to an image constructed artificially, rather than an image acquired by an image acquisition device for a scene. There are multiple different types of construct images for the above first construct image, and for specific types, reference should be made to images shown in FIG. 6 a and FIG. 6 b and the corresponding embodiments. [0032]-[0033] The above first actual characters refer to actual characters in the first construct image. The first actual characters can be obtained all at once when constructing the first construct image. [0034]-[0035] The above initial neural network model may be a neural network model that has not been trained. For example, the initial neural network model may be a convolutional neural network (CNN) model, or a recurrent neural network (RNN) model. [0086] In a first implementation, the model parameters of the training auxiliary model are adjusted to the model parameters of the trained model to be trained.)
1. generate a target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second target model, the second information being identical to or related to the input information; / 9. generating a target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second model, the second information being identical to or related to the input information; / 10. and the trained model comprising a base model which includes first model and which is trained using first information, the first information being identical to or related to the input information (Xu: [0037] When training the initial neural network model based on the first construct image and the first actual characters, the first actual characters can be used as supervision information to carry out supervised training. In this way, the pre-trained model obtained after the supervised training learns the ability to perform character recognition on images. The process of training the initial neural network model based on the first construct image and the first actual characters can be referred to as the pre-training process. Compared with the initial neural network model that has not been pre-trained, the pre-trained model can quickly and accurately process the scene image, the second construct image, and the third construct image based on the learned character recognition ability, thereby shortening the training duration of the model to be trained and improving the training efficiency. [0045] At block S102, a scene image, second actual characters in the scene image and a second construct image are obtained. [0046] The scene image refers to an image obtained by image acquisition for a real scene. The real scene corresponding to the scene image is an application scene of the model obtained by training in the subsequent actual application process, so the above real scene corresponds to the application scene of the model obtained by training. [0047] For example, if training is required to obtain a model that is applied to a road scene and capable of performing character recognition on a vehicle license plate image, the above scene image is a vehicle license plate image in the above road scene. If training is required to obtain a model that is applied to an education scene and capable of performing character recognition on a book image, the above scene image is a book image in the above education scene. [0048] The second actual characters refer to actual characters in the scene image. The second actual characters can be obtained by manual annotation. [0049] The second construct image refers to an image constructed artificially, rather than an image acquired by an image acquisition device for a scene. [0050] The characters in the second construct image are the same as the second actual characters. Taking FIGS. 2 b and 2 c as examples, the image shown in FIG. 2 b is a scene image, and the image shown in FIG. 2 c is a second construct image. The scene image shown in FIG. 2 b is an image obtained by image collection on an invoice in a financial scene. In the above image, “1490984” represents the number of the invoice, which is the second actual characters in the scene image. The second construct image shown in FIG. 2 b includes the characters of “1490984”, which are the same as the second actual characters. [0089] In the second implementation, fusion model parameters are obtained by fusing the model parameters of the trained model to be trained and the model parameters of the training auxiliary model, and the model parameters of the training auxiliary model are adjusted to the fusion model parameters.)
1. acquire an adapter trained using at least third information in a state where the adapter is coupled to the base model, the third information being identical to or related to the input information; / 9. acquiring an adapter trained using at least third information in a state where the adapter is coupled to the base model, the third information being identical to or related to the input information; / 10. target model which includes a first target model and a second target model and which is trained using second information in a state where the first base model being used as the first target model is coupled to the second target model, the second information representing the recognition target; (Xu: [0092] The fusion model parameters are obtained by fusing the model parameters of the trained model to be trained and the model parameters of the training auxiliary model, and the fusion model parameters are not only related to the model parameters of the model to be trained, but also related to the model parameters of the training auxiliary model. When adjusting the model parameters of the training auxiliary model based on the fusion model parameters, the adjusted parameters are relevant to the model parameters of the training auxiliary model itself, and the model parameters of the training auxiliary model do not need to be substantially adjusted, to achieve smooth transition for adjusting the above model parameters. [0093] At block S307, the training auxiliary model is trained after adjusting the model parameters based on a third construct image and third actual characters in the third construct image; in response to the training auxiliary model satisfying training end conditions, step S302 is repeated, to retrain the model to be trained. [0094] The third construct image may be the same image as the second construct image. In this case, the second construct image may be determined as the third construct image, and the first actual characters may be determined as the third actual characters. [0095] The third construct image may also be an image different from the second construct image. In this case, it is necessary to obtain the third construct image and the third actual characters in the third construct image. [0096] When obtaining the third construct image and the third actual characters, the third construct image and the third actual characters in the third construct image may be obtained from a pre-stored construct image library. An image generation algorithm may also be used to generate an image as the third construct image, and the actual characters in the generated image are determined as the third actual characters.)
1. and generate the trained model by coupling the adapter to the target model. / 9. and generating the trained model by coupling the adapter to the target model. / 10. and an adapter trained using at least third information as in a state the adapter is coupled to the base model, the adapter being coupled to the target model. (Xu: [0097] When training the training auxiliary model after adjusting the model parameters, the third construct image can be input into the training auxiliary model, to obtain the recognition characters that are output by the training auxiliary model. The loss value of the training auxiliary model for character recognition may be calculated according to the recognition characters and the third actual characters, and the model parameters of the training auxiliary model are adjusted according to the loss value. If the training end conditions are not satisfied, the third construct image and the third actual characters are re-obtained, and the above process is repeated until the third end conditions are satisfied, so as to realize the training of the training auxiliary model after adjusting the model parameters)
It would have been obvious before the effective filing date of the claimed invention to one of ordinary skill in the art to modify the MLM training algorithm of Rao with the improvements suggested by Xu as they are both directed towards machine learning computer vision models. The determination of obviousness is predicated upon the following findings: One skilled in the art would have been motivated to modify Rao in order to improve the overall accuracy in vision-based image processing and recognition using Xu’s algorithm for machine learning to incorporate Sim2Real machine learning models and to enhance the overall recognition. Furthermore, the prior art collectively includes each element claimed (though not all in the same reference), and one of ordinary skill in the art could have combined the elements in the manner explained above using known engineering design, interface and programming techniques, without changing a “fundamental” operating principle of Rao, while the teaching of Xu continues to perform the same function as originally taught prior to being combined, in order to produce the repeatable and predictable result of leveraging an enhanced ML model for ensuring accuracy. It is for at least the aforementioned reasons that the examiner has reached a conclusion of obviousness with respect to the claim in question.
Consider Claim 2.
The combination of Rao and Xu teaches:
2. (Currently Amended) The trained model generating device according to claim 1, wherein the adapter is coupled to an input side of the target model, and is configured to convert the input information prior to inputting the input information to the target model. (Rao: [0037] In some implementations, vision-based robot task engine(s) 158 can apply a simulation-to-real-to-simulation (“Sim2Real2Sim”) image triple of [simulated image, predicted real image, cycled simulated image] as input across vision-based robot task model(s) 190 (e.g., an RL-network) to generate a predicted value for each of the images in the Sim2Real2Sim image triple. The simulated image of the image triple can be generated by a robotic simulator 152 during performance of a simulated episode of a robotic task for which the vision-based robot task model(s) 190 is being trained. The predicted real image of the Sim2Real2Sim image triple is generated by Sim2Real engine(s) 154 processing the simulated image, using a Sim2Real model stored in the Sim2Real and Real2Sim model(s) database 180, and the predicted simulated image of the image triple is generated by Real2Sim engine(s) 156 processing the predicted real image, using a Real2Sim model stored in the Sim2Real and Real2Sim model(s) database 180. Xu: [0037] When training the initial neural network model based on the first construct image and the first actual characters, the first actual characters can be used as supervision information to carry out supervised training. In this way, the pre-trained model obtained after the supervised training learns the ability to perform character