DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This action is responsive to the amendments and remarks received 29 April 2026. Claims 1 - 20 are currently pending.
Response to Arguments
Applicant's arguments filed 29 April 2026 have been fully considered but they are not persuasive.
On pages 6 - 8 of the remarks the Applicant’s Representative argues that the instant claims are allowable over the previously cited prior art references at least because the cited combination of Shibata et al. and Wu et al. “has not been shown to teach or otherwise render obvious” the subject matter as recited in independent claims 1, 8 and 15. In particular, the Applicant’s Representative argues that the Office failed “to identify a teaching or a suggestion in Shibata as cited of a simulator using the same sensor information that is to be adjusted by a neural network to generate the travel simulation”, i.e., that Shibata et al. do not teach or suggest “one or more neural networks to use first sensor information from one or more simulations of… device to adjust second sensor information of the… device, the one or more simulations generated by one or more simulators using the second sensor information.” The Applicant’s Representative argues that the “Applicant identified only a single teaching of a simulation in the entirety of Shibata” and that “there is no additional teaching identified in Shibata as cited of how the travel simulation is generated.” Furthermore, the Applicant’s Representative argues that Wu et al. do not teach or otherwise render obvious the aforementioned disputed claim limitation(s) at least because Wu et al. “as cited has not been shown to teach or otherwise render obvious a simulator or simulations whatsoever”. Therefore, the Applicant’s Representative argues that the cited combination of Shibata et al. and Wu et al. “has not been shown to teach or otherwise render obvious” the subject matter as recited in independent claims 1, 8 and 15.
The Examiner respectfully disagrees.
The Examiner asserts that, at least, Shibata et al. disclose using “one or more neural networks to use first sensor information from one or more simulations of… device to adjust second sensor information of the… device, the one or more simulations generated by one or more simulators using the second sensor information”, see at least figures 2A - 3B, 5, 9, 10 and 12, page 2 paragraphs 0030 - 0031 and 0040, page 3 paragraphs 0051 and 0060 - 0061, page 4 paragraphs 0070 - 0074, page 5 paragraphs 0088 - 0089, page 6 paragraphs 0104 - 0105 and 0115, page 7 paragraph 0121, page 12 paragraphs 0203 - 0204 and 0212 - 0215, page 13 paragraphs 0219 and 0229 - 0233, page 14 paragraphs 0248 - 0254, page 15 paragraphs 0271 - 0272 and page 18 paragraph 0314 of Shibata et al. wherein they disclose that “it is possible to replace the noise portion in the captured image in which it is determined by the noise determination unit 12 that noise occurs using only the captured image” [0061], that “the data replacement unit 13 estimates a captured image in which no noise occurs on the basis of the captured image in which it is determined that noise occurs, and thereby generates replacement data. Then, the data replacement unit 13 replaces the noise portion of the captured image with the generated replacement data” [0070], that “noise DB 16 may store, as initial data, a captured image generated when a travel simulation is performed for each vehicle type, or a captured image acquired from the camera 21 during test travel” [0104], that “the data replacement unit 13 may generate replacement data on the basis of the captured image stored in the noise DB 16 when performing replacement. For example, in a case where the data replacement unit 13 estimates that no object is detected in the noise portion of the captured image in which it is determined by the noise determination unit 12 that noise occurs, the data replacement unit 13 extracts initial data portion corresponding to the noise portion, and generates the initial data portion as the replacement data. Furthermore, for example, in a case where the data replacement unit 13 estimates that an object is detected in the noise portion of the captured image in which it is determined by the noise determination unit 12 that noise occurs, the data replacement unit 13 extracts the initial data portion corresponding to the noise portion, superimposes the object estimated to have been detected on the initial data portion, and thereby generates the replacement data” [0105], that “a sensor noise removal device performs noise determination and replacement on the basis of a trained model in machine learning” [0204], that “first replacement-function machine learning model 3021 is a machine learning model that receives sensor data in which noise occurs as an input and outputs sensor data in which a noise portion of the sensor data in which noise occurs has been replaced with sensor data in which no noise occurs” [0214] and that “the data replacement unit 13a acquires sensor data in which a noise portion of the sensor data has been replaced with sensor data in which no noise occurs, by using the second machine learning model 302. In this manner, the data replacement unit 13a replaces the sensor data in which it is determined by the noise determination unit 12a that noise occurs. In the third embodiment, the data replacement unit 13a acquires, for a captured image in which it is determined by the noise determination unit 12 that noise occurs, a captured image in which a noise portion has been replaced with pixels in which no noise occurs” [0219].
The Examiner asserts that, as shown herein above and in the cited portions, Shibata et al. disclose that a captured image generated when a travel simulations is performed may be stored as initial data, that a data replacement unit may estimate that an object is detected in a noise portion of a captured image, extract the initial data portion corresponding to the noise portion and superimpose the object estimated to have been detected on the initial data portion to generate replacement data, that the data replacement unit replaces a noise portion of a captured image with the generated replacement data, that noise determination and replacement may be performed on the basis of a trained model in machine learning and that a machine learning model may receive sensor data in which noise occurs as an input and output sensor data in which a noise portion of the sensor data in which noise occurs has been replaced with sensor data in which no noise occurs. The Examiner asserts that Shibata et al. disclose the aforementioned disputed claim limitation(s) at least because Shibata et al. disclose inputting sensor data in which noise occurs to a trained machine learning model to generate output sensor data in which a noise portion of the sensor data in which noise occurs has been replaced with sensor data in which no noise occurs. The Examiner asserts that the trained machine learning model of Shibata et al. corresponds to the claimed one or more simulators, that the input sensor data corresponds to the claimed second sensor information and that the replaced noise portion of the sensor data in which no noise occurs corresponds to the claimed first sensor information. Thus, the Examiner asserts that at least Shibata et al. disclose the aforementioned disputed claim limitation(s). Additionally, the Examiner asserts that Shibata et al. disclose that a captured image generated when a travel simulation is performed may be stored as initial data, that a data replacement unit may estimate that an object is detected in a noise portion of a captured image (second sensor information), extract the initial data portion corresponding to the noise portion, superimpose the object estimated to have been detected on the initial data portion to generate replacement data, (first sensor information from one or more simulations generated by one or more simulators using the second sensor information) and replace a noise portion of the captured image with the generated replacement data (adjust the second sensor information). The Examiner asserts that the process of estimating that an object is detected in a noise portion of a captured image, extracting the initial data portion corresponding to the noise portion and superimposing the object estimated to have been detected on the initial data portion to generate the replacement data relies upon the second sensor information to generate the replacement data and thus corresponds to using first sensor information from one or more simulations to adjust second sensor information, the one or more simulations generated by one or more simulators using the second sensor information.
Therefore, the Examiner asserts that at least Shibata et al. disclose the aforementioned disputed claim limitation(s) and that Shibata et al. in view of Wu et al. disclose and render obvious the subject matter as recited in independent claims 1, 8 and 15.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1 - 20 are rejected under 35 U.S.C. 103 as being unpatentable over Shibata et al. U.S. Publication No. 2023/0325983 A1 in view of Wu et al. U.S. Publication No. 2020/0293064 A1.
- With regards to claim 1, Shibata et al. disclose one or more processors, (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) comprising: one or more circuits (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) to use one or more neural networks to use first sensor information from one or more simulations of a device (Shibata et al., Figs. 2A - 3B, 5, 9 & 10, Pg. 2 ¶ 0025 - 0026, 0031, 0035 - 0038 and 0040, Pg. 3 ¶ 0051, Pg. 5 ¶ 0088 - 0089, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 12 ¶ 0203 - 0205 and 0212 - 0215, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 17 ¶ 0307) to adjust second sensor information of the device, (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315) the one or more simulations generated by one or more simulators using the second sensor information. (Shibata et al., Figs. 2A - 3B, 5, 9, 10 & 12, Pg. 3 ¶ 0051 and 0060 - 0061, Pg. 4 ¶ 0066 - 0068 and 0070 - 0074, Pg. 5 ¶ 0088 - 0089, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 7 ¶ 0121, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 and 0229 - 0233, Pg. 14 ¶ 0248 - 0254, Pg. 15 ¶ 0271 - 0272, Pg. 18 ¶ 0314) Shibata et al. fail to disclose explicitly an autonomous device. Pertaining to analogous art, Wu et al. disclose using one or more neural networks to use first sensor information from one or more simulations of an autonomous device to adjust second sensor information of the autonomous device, (Wu et al., Abstract, Figs. 1 - 4, 7, 8 & 14A - 14D, Pg. 1 ¶ 0002 and 0004 - 0006, Pg. 2 ¶ 0027 - Pg. 3 ¶ 0033, Pg. 3 ¶ 0035, Pg. 4 ¶ 0041 and 0044, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 13 ¶ 0129 - 0130, Pg. 14 ¶ 0135 and 0138, Pg. 18 ¶ 0171, Pg. 20 ¶ 0190 - 0193, Pg. 24 ¶ 0232) the one or more simulations generated by one or more simulators using the second sensor information. (Wu et al., Figs. 1 - 4, 7 - 9, 11 & 13, Pg. 2 ¶ 0028 - Pg. 3 ¶ 0034, Pg. 4 ¶ 0041, Pg. 4 ¶ 0044 - Pg. 5 ¶ 0045, Pg. 5 ¶ 0048 - 0052, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 10 ¶ 0094, Pg. 11 ¶ 0101 - 0106, Pg. 13 ¶ 0129 - 0130, Pg. 18 ¶ 0171, Pg. 24 ¶ 0232 [“server(s) 1478 may be used to train machine learning models (e.g., neural networks) based on training data. The training data may be generated by the vehicles, and/or may be generated in a simulation (e.g., using a game engine). In some examples, the training data is tagged (e.g., where the neural network benefits from supervised learning) and/or undergoes other pre-processing, while in other examples the training data is not tagged and/or pre-processed (e.g., where the neural network does not require supervised learning).”]) Shibata et al. and Wu et al. are combinable because they are both directed towards image processing systems that utilize neural networks to process image data captured from vehicle-mounted cameras. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Shibata et al. with the teachings of Wu et al. This modification would have been prompted in order to enhance the base device of Shibata et al. with the well-known and applicable technique Wu et al. applied to a comparable device. Adjusting sensor information of an autonomous device, as taught by Wu et al., would enhance the base device of Shibata et al. by allowing for it to improve the reliability of sensor data obtained and/or employed in a wider variety of situations and for it to be utilized in an increased number and variety of related and applicable applications and/or environments, such as autonomous driving functions, thereby improving its overall appeal, usefulness and marketability to potential end-users. Furthermore, this modification would have been prompted by the teachings and suggestions of Shibata et al. that their teachings may be applied to sensor data acquired from sensors mounted on a vehicle, that the sensor data may be used for various types of processing related to the vehicle including processing performed using artificial intelligence, that data generated during a travel simulation of the vehicle may be utilized to generate replacement data and that neural networks may be used to generate the replacement data, see at least page 2 paragraphs 0025 - 0026 and 0030, page 6 paragraphs 0104 - 0105, page 12 paragraphs 0203 - 0204 and 0212 - 0215, page 14 paragraphs 0248 - 0256 and page 18 paragraph 0315 of Shibata et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the base device of Shibata et al. would be utilized in connection with sensor data acquired from an autonomous device so as to allow for the base device of Shibata et al. to be utilized in an increased number of applications and/or environments and improve the reliability of sensor data obtained and/or employed in a wider variety of situations so as to improve its overall appeal, usefulness and marketability to potential end-users. Therefore, it would have been obvious to combine Shibata et al. with Wu et al. to obtain the invention as specified in claim 1.
- With regards to claim 2, Shibata et al. in view of Wu et al. disclose the one or more processors of claim 1, wherein the one or more circuits (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to replace one or more portions of the second sensor information. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 3, Shibata et al. in view of Wu et al. disclose the one or more processors of claim 1, wherein: the one or more circuits (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are further to label one or more first portions of the second sensor information as including validated data and one or more second portions of the second sensor information as including unvalidated data; (Shibata et al., Abstract, Figs. 2A - 5 & 12, Pg. 1 ¶ 0008, Pg. 3 ¶ 0048 - 0051 and 0055 - 0058, Pg. 5 ¶ 0081 - 0085, Pg. 8 ¶ 0137 - 0138 and 0141 - 0143, Pg. 12 ¶ 0203, 0213 and 0218, Pg. 15 ¶ 0267 - 0273 [Noise portions of the sensor data, image, in Shibata et al. correspond to one or more second portions including unvalidated data and the remaining portions of the sensor data, image, correspond to one or more first portions including validated data.]) and the one or more circuits (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to adjust the one or more second portions. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 4, Shibata et al. in view of Wu et al. disclose the one or more processors of claim 1, wherein the one or more circuits (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are further to identify the second sensor information as comprising corrupted data. (Shibata et al., Abstract, Figs. 2A - 5 & 12, Pg. 1 ¶ 0008, Pg. 3 ¶ 0048 - 0051 and 0055 - 0058, Pg. 5 ¶ 0081 - 0085, Pg. 8 ¶ 0137 - 0138 and 0141 - 0143, Pg. 12 ¶ 0203, 0213 and 0218, Pg. 15 ¶ 0267 - 0273)
- With regards to claim 5, Shibata et al. in view of Wu et al. disclose the one or more processors of claim 1, wherein the one or more circuits (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are further to receive the one or more simulations as output from the one or more simulators. (Shibata et al., Figs. 1, 4 - 6B, 9, 10, 12 & 13, Pg. 2 ¶ 0030 - 0032 and 0040, Pg. 3 ¶ 0050 - 0051 and 0061, Pg. 4 ¶ 0070, Pg. 5 ¶ 0088 - 0092, Pg. 6 ¶ 0098 - 0105 and 0115 - 0117, Pg. 8 ¶ 0146 - 0151, Pg. 12 ¶ 0203 - 0204 and 0210 - 0215, Pg. 13 ¶ 0225 and 0229 - 0233, Pg. 14 ¶ 0248 - 0253, Pg. 15 ¶ 0268 - 0275, Pg. 16 ¶ 0279 - 0283) In addition, Wu et al. disclose wherein the one or more circuits (Wu et al., Figs. 14C - 15, Pg. 14 ¶ 0134 - 0136, Pg. 15 ¶ 0147 - Pg. 16 ¶ 0157, Pg. 18 ¶ 0177 - Pg. 19 ¶ 0189, Pg. 23 ¶ 0230 - Pg. 24 ¶ 0238) are further to receive the one or more simulations as output from the one or more simulators. (Wu et al., Figs. 1 - 4, 7 - 9, 11 & 13, Pg. 2 ¶ 0028 - Pg. 3 ¶ 0034, Pg. 4 ¶ 0041, Pg. 4 ¶ 0044 - Pg. 5 ¶ 0045, Pg. 5 ¶ 0048 - 0052, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 10 ¶ 0094, Pg. 11 ¶ 0101 - 0106, Pg. 18 ¶ 0171, Pg. 24 ¶ 0232 [“server(s) 1478 may be used to train machine learning models (e.g., neural networks) based on training data. The training data may be generated by the vehicles, and/or may be generated in a simulation (e.g., using a game engine). In some examples, the training data is tagged (e.g., where the neural network benefits from supervised learning) and/or undergoes other pre-processing, while in other examples the training data is not tagged and/or pre-processed (e.g., where the neural network does not require supervised learning).”])
- With regards to claim 6, Shibata et al. in view of Wu et al. disclose the one or more processors of claim 1, wherein the one or more circuits (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to adjust a smoothness of the second sensor information. (Shibata et al., Figs. 2A - 3B, 9 & 10, Pg. 2 ¶ 0032 and 0040, Pg. 4 ¶ 0070 - 0073, Pg. 5 ¶ 0081 - 0083 and 0088 - 0093, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 7 ¶ 0135 - Pg. 8 ¶ 0138, Pg. 9 ¶ 0156 - 0158, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273) In addition, analogous art Wu et al. disclose using the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to adjust a smoothness of the second sensor information. (Wu et al., Pg. 2 ¶ 0032, Pg. 3 ¶ 0038 - Pg. 4 ¶ 0041, Pg. 4 ¶ 0043, Pg. 5 ¶ 0045 - 0046, Pg. 19 ¶ 0183 - 0185)
- With regards to claim 7, Shibata et al. in view of Wu et al. disclose the one or more processors of claim 1, wherein the one or more circuits (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to modify one or more first portions of the second sensor information and maintain one or more second portions of the second sensor information. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 8, Shibata et al. disclose a system, (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 1 ¶ 0008, Pg. 2 ¶ 0024 - 0025 and 0043, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 12 ¶ 0203 - 0205 and 0208 - 0211, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0285 - Pg. 17 ¶ 0295) comprising: one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) to use one or more neural networks to use first sensor information from one or more simulations of a device (Shibata et al., Figs. 2A - 3B, 5, 9 & 10, Pg. 2 ¶ 0025 - 0026, 0031, 0035 - 0038 and 0040, Pg. 3 ¶ 0051, Pg. 5 ¶ 0088 - 0089, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 12 ¶ 0203 - 0205 and 0212 - 0215, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 17 ¶ 0307) to adjust second sensor information of the device, (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315) the one or more simulations generated by one or more simulators using the second sensor information. (Figs. 2A - 3B, 5, 9, 10 & 12, Pg. 3 ¶ 0051 and 0060 - 0061, Pg. 4 ¶ 0066 - 0068 and 0070 - 0074, Pg. 5 ¶ 0088 - 0089, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 7 ¶ 0121, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 and 0229 - 0233, Pg. 14 ¶ 0248 - 0254, Pg. 15 ¶ 0271 - 0272, Pg. 18 ¶ 0314) Shibata et al. fail to disclose explicitly an autonomous device. Pertaining to analogous art, Wu et al. disclose using one or more neural networks to use first sensor information from one or more simulations of an autonomous device to adjust second sensor information of the autonomous device, (Wu et al., Abstract, Figs. 1 - 4, 7, 8 & 14A - 14D, Pg. 1 ¶ 0002 and 0004 - 0006, Pg. 2 ¶ 0027 - Pg. 3 ¶ 0033, Pg. 3 ¶ 0035, Pg. 4 ¶ 0041 and 0044, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 13 ¶ 0129 - 0130, Pg. 14 ¶ 0135 and 0138, Pg. 18 ¶ 0171, Pg. 20 ¶ 0190 - 0193, Pg. 24 ¶ 0232) the one or more simulations generated by one or more simulators using the second sensor information. (Wu et al., Figs. 1 - 4, 7 - 9, 11 & 13, Pg. 2 ¶ 0028 - Pg. 3 ¶ 0034, Pg. 4 ¶ 0041, Pg. 4 ¶ 0044 - Pg. 5 ¶ 0045, Pg. 5 ¶ 0048 - 0052, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 10 ¶ 0094, Pg. 11 ¶ 0101 - 0106, Pg. 13 ¶ 0129 - 0130, Pg. 18 ¶ 0171, Pg. 24 ¶ 0232 [“server(s) 1478 may be used to train machine learning models (e.g., neural networks) based on training data. The training data may be generated by the vehicles, and/or may be generated in a simulation (e.g., using a game engine). In some examples, the training data is tagged (e.g., where the neural network benefits from supervised learning) and/or undergoes other pre-processing, while in other examples the training data is not tagged and/or pre-processed (e.g., where the neural network does not require supervised learning).”]) Shibata et al. and Wu et al. are combinable because they are both directed towards image processing systems that utilize neural networks to process image data captured from vehicle-mounted cameras. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Shibata et al. with the teachings of Wu et al. This modification would have been prompted in order to enhance the base device of Shibata et al. with the well-known and applicable technique Wu et al. applied to a comparable device. Adjusting sensor information of an autonomous device, as taught by Wu et al., would enhance the base device of Shibata et al. by allowing for it to improve the reliability of sensor data obtained and/or employed in a wider variety of situations and for it to be utilized in an increased number and variety of related and applicable applications and/or environments, such as autonomous driving functions, thereby improving its overall appeal, usefulness and marketability to potential end-users. Furthermore, this modification would have been prompted by the teachings and suggestions of Shibata et al. that their teachings may be applied to sensor data acquired from sensors mounted on a vehicle, that the sensor data may be used for various types of processing related to the vehicle including processing performed using artificial intelligence, that data generated during a travel simulation of the vehicle may be utilized to generate replacement data and that neural networks may be used to generate the replacement data, see at least page 2 paragraphs 0025 - 0026 and 0030, page 6 paragraphs 0104 - 0105, page 12 paragraphs 0203 - 0204 and 0212 - 0215, page 14 paragraphs 0248 - 0256 and page 18 paragraph 0315 of Shibata et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the base device of Shibata et al. would be utilized in connection with sensor data acquired from an autonomous device so as to allow for the base device of Shibata et al. to be utilized in an increased number of applications and/or environments and improve the reliability of sensor data obtained and/or employed in a wider variety of situations so as to improve its overall appeal, usefulness and marketability to potential end-users. Therefore, it would have been obvious to combine Shibata et al. with Wu et al. to obtain the invention as specified in claim 8.
- With regards to claim 9, Shibata et al. in view of Wu et al. disclose the system of claim 8, wherein the one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to generate data to replace one or more portions of the second sensor information. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 10, Shibata et al. in view of Wu et al. disclose the system of claim 8, wherein: the one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are further to generate a first set of labels identifying one or more first portions of the second sensor information as including validated data and a second set of labels identifying one or more second portions of the second sensor information as including unvalidated data; (Shibata et al., Abstract, Figs. 2A - 5 & 12, Pg. 1 ¶ 0008, Pg. 3 ¶ 0048 - 0051 and 0055 - 0058, Pg. 5 ¶ 0081 - 0085, Pg. 8 ¶ 0137 - 0138 and 0141 - 0143, Pg. 12 ¶ 0203, 0213 and 0218, Pg. 15 ¶ 0267 - 0273 [Noise portions of the sensor data, image, in Shibata et al. correspond to one or more second portions labeled as unvalidated data and the remaining portions of the sensor data, image, correspond to one or more first portions labeled as validated data.]) and the one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information based, at least in part, on the first and second sets of labels. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 11, Shibata et al. in view of Wu et al. disclose the system of claim 8, wherein: the one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are further to identify the second sensor information as comprising corrupted data; (Shibata et al., Abstract, Figs. 2A - 5 & 12, Pg. 1 ¶ 0008, Pg. 3 ¶ 0048 - 0051 and 0055 - 0058, Pg. 5 ¶ 0081 - 0085, Pg. 8 ¶ 0137 - 0138 and 0141 - 0143, Pg. 12 ¶ 0203, 0213 and 0218, Pg. 15 ¶ 0267 - 0273) and the one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to remove the corrupted data from the second sensor information. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 12, Shibata et al. in view of Wu et al. disclose the system of claim 8, wherein the one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are further to receive the one or more simulations, from the one or more simulators, to be provided as input to the one or more neural networks. (Shibata et al., Figs. 1, 4 - 6B, 9, 10, 12 & 13, Pg. 2 ¶ 0030 - 0032 and 0040, Pg. 3 ¶ 0050 - 0051 and 0061, Pg. 5 ¶ 0088 - 0092, Pg. 6 ¶ 0098 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0210 - 0215, Pg. 13 ¶ 0225 and 0229 - 0233, Pg. 14 ¶ 0248 - 0253, Pg. 15 ¶ 0271 - 0275, Pg. 16 ¶ 0279 - 0283) In addition, Wu et al. disclose wherein the one or more processors (Wu et al., Figs. 14C - 15, Pg. 14 ¶ 0134 - 0136, Pg. 15 ¶ 0147 - Pg. 16 ¶ 0157, Pg. 18 ¶ 0177 - Pg. 19 ¶ 0189, Pg. 23 ¶ 0230 - Pg. 24 ¶ 0238) are further to receive the one or more simulations, from the one or more simulators, to be provided as input to the one or more neural networks. (Wu et al., Figs. 1 - 4, 7 - 9, 11 & 13, Pg. 2 ¶ 0028 - Pg. 3 ¶ 0034, Pg. 4 ¶ 0041, Pg. 4 ¶ 0044 - Pg. 5 ¶ 0045, Pg. 5 ¶ 0048 - 0052, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 10 ¶ 0094, Pg. 11 ¶ 0101 - 0106, Pg. 18 ¶ 0171, Pg. 24 ¶ 0232 [“server(s) 1478 may be used to train machine learning models (e.g., neural networks) based on training data. The training data may be generated by the vehicles, and/or may be generated in a simulation (e.g., using a game engine). In some examples, the training data is tagged (e.g., where the neural network benefits from supervised learning) and/or undergoes other pre-processing, while in other examples the training data is not tagged and/or pre-processed (e.g., where the neural network does not require supervised learning).”])
- With regards to claim 13, Shibata et al. in view of Wu et al. disclose the system of claim 8, wherein the one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to increase a smoothness across the second sensor information. (Shibata et al., Figs. 2A - 3B, 9 & 10, Pg. 2 ¶ 0032 and 0040, Pg. 4 ¶ 0070 - 0073, Pg. 5 ¶ 0081 - 0083 and 0088 - 0093, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 7 ¶ 0135 - Pg. 8 ¶ 0138, Pg. 9 ¶ 0156 - 0158, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273) In addition, analogous art Wu et al. disclose using the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to increase a smoothness across the second sensor information. (Wu et al., Pg. 2 ¶ 0032, Pg. 3 ¶ 0038 - Pg. 4 ¶ 0041, Pg. 4 ¶ 0043, Pg. 5 ¶ 0045 - 0046, Pg. 19 ¶ 0183 - 0185)
- With regards to claim 14, Shibata et al. in view of Wu et al. disclose the system of claim 8, wherein the one or more processors (Shibata et al., Figs. 1, 6A, 6B, 9 & 10, Pg. 8 ¶ 0146 - Pg. 9 ¶ 0151, Pg. 16 ¶ 0285 - 0287, Pg. 16 ¶ 0290 - Pg. 17 ¶ 0295) are to use the one or more neural networks to adjust the second sensor information at least by using the one or more neural networks to modify one or more first portions of the second sensor information and maintain one or more second portions of the second sensor information. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 15, Shibata et al. disclose a method, (Shibata et al., Abstract, Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 3 2 0044 - 0051, Pg. 6 ¶ 0108 - 0118, Pg. 12 ¶ 0203 - 0205 and 0212 - 0215, Pg. 15 ¶ 0263 - 0273, Pg. 16 ¶ 0287 and 0294) comprising: causing one or more simulators to use second sensor information of a device to generate one or more simulations of the device; (Shibata et al., Figs. 2A - 3B, 5, 9, 10 & 12, Pg. 2 ¶ 0025, Pg. 3 ¶ 0051 and 0060 - 0061, Pg. 4 ¶ 0066 - 0068 and 0070 - 0074, Pg. 5 ¶ 0088 - 0095, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 7 ¶ 0121, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 and 0229 - 0235, Pg. 14 ¶ 0248 - 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315) and using one or more neural networks to use first sensor information, from the one or more simulations, (Shibata et al., Figs. 2A - 3B, 5, 9 & 10, Pg. 2 ¶ 0025 - 0026, 0031, 0035 - 0038 and 0040, Pg. 3 ¶ 0051, Pg. 5 ¶ 0088 - 0089, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 12 ¶ 0203 - 0205 and 0212 - 0215, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 17 ¶ 0307) to adjust the second sensor information. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315) Shibata et al. fail to disclose explicitly an autonomous device. Pertaining to analogous art, Wu et al. disclose causing one or more simulators to use second sensor information of an autonomous device to generate one or more simulations of the autonomous device; (Wu et al., Figs. 1 - 4, 7 - 9, 11 & 13, Pg. 2 ¶ 0028 - Pg. 3 ¶ 0034, Pg. 4 ¶ 0041, Pg. 4 ¶ 0044 - Pg. 5 ¶ 0045, Pg. 5 ¶ 0048 - 0052, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 10 ¶ 0094, Pg. 11 ¶ 0101 - 0106, Pg. 13 ¶ 0129 - 0130, Pg. 18 ¶ 0171, Pg. 24 ¶ 0232 [“server(s) 1478 may be used to train machine learning models (e.g., neural networks) based on training data. The training data may be generated by the vehicles, and/or may be generated in a simulation (e.g., using a game engine). In some examples, the training data is tagged (e.g., where the neural network benefits from supervised learning) and/or undergoes other pre-processing, while in other examples the training data is not tagged and/or pre-processed (e.g., where the neural network does not require supervised learning)”]) and using one or more neural networks to use first sensor information, from the one or more simulations, to adjust the second sensor information. (Wu et al., Abstract, Figs. 1 - 4, 7, 8 & 14A - 14D, Pg. 1 ¶ 0002 and 0004 - 0006, Pg. 2 ¶ 0027 - Pg. 3 ¶ 0035, Pg. 4 ¶ 0041 and 0044, Pg. 5 ¶ 0048 - 0052, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 13 ¶ 0129 - 0130, Pg. 14 ¶ 0135 and 0138, Pg. 18 ¶ 0171, Pg. 20 ¶ 0190 - 0193, Pg. 24 ¶ 0232) Shibata et al. and Wu et al. are combinable because they are both directed towards image processing systems that utilize neural networks to process image data captured from vehicle-mounted cameras. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Shibata et al. with the teachings of Wu et al. This modification would have been prompted in order to enhance the base device of Shibata et al. with the well-known and applicable technique Wu et al. applied to a comparable device. Adjusting sensor information of an autonomous device, as taught by Wu et al., would enhance the base device of Shibata et al. by allowing for it to improve the reliability of sensor data obtained and/or employed in a wider variety of situations and for it to be utilized in an increased number and variety of related and applicable applications and/or environments, such as autonomous driving functions, thereby improving its overall appeal, usefulness and marketability to potential end-users. Furthermore, this modification would have been prompted by the teachings and suggestions of Shibata et al. that their teachings may be applied to sensor data acquired from sensors mounted on a vehicle, that the sensor data may be used for various types of processing related to the vehicle including processing performed using artificial intelligence, that data generated during a travel simulation of the vehicle may be utilized to generate replacement data and that neural networks may be used to generate the replacement data, see at least page 2 paragraphs 0025 - 0026 and 0030, page 6 paragraphs 0104 - 0105, page 12 paragraphs 0203 - 0204 and 0212 - 0215, page 14 paragraphs 0248 - 0256 and page 18 paragraph 0315 of Shibata et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the base device of Shibata et al. would be utilized in connection with sensor data acquired from an autonomous device so as to allow for the base device of Shibata et al. to be utilized in an increased number of applications and/or environments and improve the reliability of sensor data obtained and/or employed in a wider variety of situations so as to improve its overall appeal, usefulness and marketability to potential end-users. Therefore, it would have been obvious to combine Shibata et al. with Wu et al. to obtain the invention as specified in claim 15.
- With regards to claim 16, Shibata et al. in view of Wu et al. disclose the method of claim 15, wherein using the one or more neural networks to adjust the second sensor information comprises using the one or more neural networks to generate data, based, at least in part, on the first sensor information, to replace one or more portions of the second sensor information. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 17, Shibata et al. in view of Wu et al. disclose the method of claim 15, further comprising labeling corrupted data in the second sensor information as unvalidated data, (Shibata et al., Abstract, Figs. 2A - 5 & 12, Pg. 1 ¶ 0008, Pg. 3 ¶ 0048 - 0051 and 0055 - 0058, Pg. 5 ¶ 0081 - 0085, Pg. 8 ¶ 0137 - 0138 and 0141 - 0143, Pg. 12 ¶ 0203, 0213 and 0218, Pg. 15 ¶ 0267 - 0273 [Noise portions of the sensor data, image, in Shibata et al. correspond to corrupted data labeled as unvalidated data.]) and wherein using the one or more neural networks to adjust the second sensor information comprises using the one or more neural networks to correct the labeled corrupted data. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
- With regards to claim 18, Shibata et al. in view of Wu et al. disclose the method of claim 15, wherein causing the one or more simulators to use the second sensor information to generate the one or more simulations comprises causing the one or more simulators to use the second sensor information to update one or more parameters of a simulated environment. (Shibata et al., Figs. 2A - 3B, 5, 9, 10, 12 & 13, Pg. 2 ¶ 0025, 0030 - 0032 and 0040, Pg. 3 ¶ 0060 - 0061, Pg. 4 ¶ 0066 - 0068 and 0070 - 0074, Pg. 5 ¶ 0088 - 0095, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0210 - 0215, Pg. 13 ¶ 0225 and 0229 - 0235, Pg. 14 ¶ 0248 - 0253 and 0255, Pg. 15 ¶ 0259 and 0268 - 0275, Pg. 16 ¶ 0279 - 0283, Pg. 18 ¶ 0312 - 0315 [The generated replacement data corresponding to the noise portion of the sensor data corresponds to a simulated environment.]) In addition, Wu et al. disclose wherein causing the one or more simulators to use the second sensor information to generate the one or more simulations comprises causing the one or more simulators to use the second sensor information to update one or more parameters of a simulated environment. (Wu et al., Figs. 1 - 4, 7 - 9, 11 & 13, Pg. 2 ¶ 0028 - Pg. 3 ¶ 0034, Pg. 4 ¶ 0041, Pg. 4 ¶ 0044 - Pg. 5 ¶ 0045, Pg. 5 ¶ 0048 - 0052, Pg. 8 ¶ 0080 - 0084, Pg. 9 ¶ 0086 - 0087, Pg. 10 ¶ 0094, Pg. 11 ¶ 0101 - 0106, Pg. 18 ¶ 0171, Pg. 24 ¶ 0232)
- With regards to claim 19, Shibata et al. in view of Wu et al. disclose the method of claim 15, wherein: using the one or more neural networks to adjust the second sensor information comprises using the one or more neural networks to increase a smoothness of the second sensor information. (Shibata et al., Figs. 2A - 3B, 9 & 10, Pg. 2 ¶ 0032 and 0040, Pg. 4 ¶ 0070 - 0073, Pg. 5 ¶ 0081 - 0083 and 0088 - 0093, Pg. 6 ¶ 0104 - 0105 and 0115, Pg. 7 ¶ 0135 - Pg. 8 ¶ 0138, Pg. 9 ¶ 0156 - 0158, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273) Shibata et al. fail to disclose expressly wherein: the second sensor information is a time series. In addition, Shibata et al. fail to disclose explicitly increasing a smoothness of the second sensor information with respect to time. Pertaining to analogous art, Wu et al. disclose wherein: the second sensor information is a time series, (Wu et al., Abstract, Figs. 4, 9 & 10, Pg. 2 ¶ 0030, Pg. 3 ¶ 0033 and 0035 - 0038, Pg. 5 ¶ 0048 - 0049 and 0056, Pg. 6 ¶ 0058, Pg. 10 ¶ 0096 - 0098, Pg. 11 ¶ 0105 - 0106) and using the one or more neural networks to increase a smoothness of the second sensor information with respect to time. (Wu et al., Pg. 2 ¶ 0032, Pg. 3 ¶ 0038 - Pg. 4 ¶ 0041, Pg. 4 ¶ 0043, Pg. 5 ¶ 0045 - 0046, Pg. 19 ¶ 0183 - 0185) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Shibata et al. in view of Wu et al. with additional teachings of Wu et al. This modification would have been prompted in order to enhance the combined base device of Shibata et al. in view of Wu et al. with the well-known and applicable technique Wu et al. applied to a comparable device. Adjusting a time series of sensor information and increasing a smoothness of the sensor information with respect to time, as taught by Wu et al., would enhance the combined base device by allowing for it to improve the reliability of an increased number and variety of types of sensor information that are typically employed and utilized in connection with autonomous vehicles and/or autonomous driving functions, such as sequences of captured images, and by further improving the quality and reliability of the second sensor information so as to enhance the ability of the combined base device to make accurate and reliable decisions based upon the second sensor information. Furthermore, this modification would have been prompted by the teachings and suggestions of Shibata et al. that their teachings may be applied to sensor data acquired from sensors mounted on a vehicle, that the sensor data may be used for various types of processing related to the vehicle including processing performed using artificial intelligence, that replacement data may be generated on the basis of neighboring pixel, that an average value of relevant neighboring pixels may be used to generate the replacement data and that a value equal to an adjacent pixel value may be used to generate the replacement data, see at least page 2 paragraphs 0025 - 0026 and 0030 and page 4 paragraphs 0070 - 0073 of Shibata et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that a smoothness of a time series of sensor information would be increased with respect to time so as to allow for the combined base device to improve the reliability of an increased number and variety of types of sensor information and to further improve the quality and reliability of the second sensor information so as to enhance the ability of the combined base device to make accurate and reliable decisions based upon the second sensor information. Therefore, it would have been obvious to combine Shibata et al. in view of Wu et al. with additional teachings of Wu et al. to obtain the invention as specified in claim 19.
- With regards to claim 20, Shibata et al. in view of Wu et al. disclose the method of claim 15, wherein using the one or more neural networks to adjust the second sensor information comprises using the one or more neural networks to modify one or more first portions of the second sensor information and maintain one or more second portions of the second sensor information. (Shibata et al., Figs. 1 - 5, 9, 10 & 12, Pg. 1 ¶ 0008, Pg. 2 ¶ 0025 - 0026, 0029 - 0032 and 0040, Pg. 3 ¶ 0051, Pg. 4 ¶ 0070 - 0074, Pg. 5 ¶ 0081, Pg. 6 ¶ 0104 - 0105 and 0115 - 0117, Pg. 12 ¶ 0203 - 0204 and 0212 - 0215, Pg. 13 ¶ 0219 - 0221, Pg. 14 ¶ 0248, 0252 and 0255, Pg. 15 ¶ 0259 and 0270 - 0273, Pg. 18 ¶ 0312 - 0315)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC RUSH whose telephone number is (571) 270-3017. The examiner can normally be reached 9am - 5pm Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270 - 5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ERIC RUSH/Primary Examiner, Art Unit 2677
Prosecution Insights