DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
1. Claims 1, 3, and 5-20 are currently pending.
2. Claims 2 and 4 are canceled.
Claim Rejections - 35 USC § 103
3. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
4. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
5. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
8. Claims 1, 3, and 5-20 are rejected under 35 U.S.C. 103 as being unpatentable over Yamanaka (US 20210374440 A1), in view of Parchami (US 20230177842 A1), and in further view of Neitemeier (US 20180084708 A1).
9. Regarding Claim 1, Yamanaka teaches a control system for monitoring operation of a… machine, the control system comprising one or more controllers, and being configured to: receive image data indicative of an input image of a working environment of the… machine (Yamanaka: [0013], [0022], and [0040] “on-road obstacle detection devices that detect on-road obstacles in images…”);
Encode the image data utilizing an encoder network to map the image data to a lower-dimensional feature space (Yamanaka: [0030] Note that the images being processed with an VAE is equivalent to utilizing an encoder network.);
Decode the encoded data to form a reconstructed image of the working environment (Yamanaka: [0035] Note that the VAE includes an encoded and decoder that reconstructs the image from low dimensional encoding.);
Compare the reconstructed image with the input image to generate an anomaly map by calculating, pixel-wise, a relative perceptual loss between the input image and the reconstructed image... to the input image and to the reconstructed image (Yamanaka: [0036] and [0040] “The difference between the semantically labelled image and the reconstructed image is computed in the present exemplary embodiment.”);
And generate and output one or more… signals... in dependence on the comparison (Yamanaka: [0037] and [0038] Step 106 determines if the difference is greater than a threshold and Step 108 detects divergent portions based on the difference being greater than the threshold.).
Yamanaka fails to explicitly teach applying a multi-layer artificial neural network to the input image and to the reconstructed image.
However, in the same field of endeavor, Parchami teaches to calculating… a relative perceptual loss between the input image and the reconstructed image by applying a multi-layer artificial neural network to the input image and to the reconstructed image (Parchami: [0017] and [0038] “The sparse convolutional neural network can include a plurality of convolutional layers”).
Yamanaka and Parchami are considered to be analogous to the claim invention because they are in the same field of image processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Yamanaka to incorporate the teachings of Parchami to apply a multi-layer artificial neural network to the input image and reconstructed image for comparing the reconstructed image with the input image to generate an anomaly map because it provides the benefit of improving the ability to detect objects while decreasing the time and computing resources to do so. This motivation is explicitly explained in [0013] of Parchami.
Yamanaka and Parchami fail to explicitly teach a control system for monitoring operation of an agricultural machine… and being configured to: generate and output one or more control signals for controlling operation of one or more operable components associated with the machine in dependence on the comparison.
However, in the same field of endeavor, Neitemeier teaches a control system for monitoring operation of an agricultural machine… and being configured to (Neitemeier: [0002] “This application relates generally to agricultural work machines, and specifically relates to an agricultural work machine…”):
Generate and output one or more control signals for controlling operation of one or more operable components associated with the machine in dependence on the comparison (Neitemeier: [0036] and [0067] “The at least one further processable image data may be used to control the agricultural work machine and/or its work organs…”).
Yamanaka, Parchami, and Neitemeier are considered to be analogous to the claim invention because they are in the same field of image processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Yamanaka and Parchami to incorporate the teachings of Neitemeier to generate and output control signals for controlling the operation of an agricultural machine based on the comparison because it provides the benefit of determining an anomaly in the environment of the work machine as explained in [0017] of Neitemeier. This provides the additional benefit of collision avoidance to increase the safety of the machine, passengers, and surroundings.
10. Regarding Claim 3, Yamanaka, Parchami, and Neitemeier remain applied above in Claim 1, and further, Neitemeier teaches to generate and output one or more control signals for controlling operation of the one or more operable components associated with the machine in dependence on the generated anomaly map (Neitemeier: [0036] and [0067] “the identified anomalies 19, can be used to control the agricultural work machine 2 and/or its work organs. Thus, the agricultural work machine 2 may be configured to avoid and/or stop at an anomaly 19 autonomously.”).
11. Regarding Claim 5, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 1, and further, Yamanaka teaches to determine a pixel anomaly score for each pixel within the reconstructed image (Yamanaka: [0032], [0037], and [0038] Note that comparing to a preset threshold is equivalent to determining a pixel anomaly score. Also, note that detecting any location/region where the difference is greater than a threshold is equivalent to the pixel anomaly score for each pixel within the reconstructed image.).
12. Regarding Claim 6, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 5, and further, Yamanaka teaches to determine an image anomaly score in dependence on the determined pixel anomaly scores (Yamanaka: [0032], [0037], and [0038] Note that determining if there is a region for which the difference is above a threshold value is equivalent to determining an image anomaly score in dependence of the pixel anomaly score.).
13. Regarding Claim 7, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 6, and further, Yamanaka teaches to compare the image anomaly score with a threshold anomaly score; and determine the presence of an anomaly within working environment in dependence on the comparison (Yamanaka: [0032], [0037], and [0038]).
14. Regarding Claim 8, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 7, and further, Yamanaka teaches to determine the presence of an anomaly within the working environment in dependence on the image anomaly score exceeding the threshold anomaly score (Yamanaka: [0032], [0037], and [0038] Detecting a location where the difference is greater than a threshold is equivalent to comparing the image anomaly score with a threshold.).
15. Regarding Claim 9, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 1, and further, Yamanaka teaches to utilize an autoencoder architecture for encoding the received image data and decoding the encoded data (Yamanaka: [0030] Note that the VAE encodes and decodes the data.).
16. Regarding Claim 10, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 9, and further, Yamanaka teaches wherein the autoencoder architecture comprises an encoder network and a decoder network (Yamanaka: [0026] and [0030] Note that the trained VAE that encodes and decodes the data is equivalent to an encoder network and decoder network.).
17. Regarding Claim 11, Yamanaka, Parchami, and Neitemeier remains as applied above in Claim 9, and further, Yamanaka teaches wherein the autoencoder architecture is trained utilizing a training dataset (Yamanaka: [0023] “This discriminator is trained using supervised learning in which images of normal travel environments in which on-road obstacles are not present are gathered…”).
18. Regarding Claim 12, Yamanaka, Parchami, and Neitemeier remains as applied above in Claim 11, and further, Yamanaka teaches wherein the training dataset comprises a normal dataset comprising image data relating to images obtained during normal operation of the agricultural machine within a working environment (Yamanaka: [0023] “This discriminator is trained using supervised learning in which images of normal travel environments in which on-road obstacles are not present are gathered…”).
19. Regarding Claim 13, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 1, and further, Neitemeier teaches wherein the one or more operable components comprise a user interface associated with the machine (Neitemeier: [0067] data output unit 21).
20. Regarding Claim 14, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 13, and further, Neitemeier teaches operable to present, on the user interface, a representation of an anomaly map to the operator to inform the operator of a detected anomaly within the working environment of the machine (Neitemeier: [0038] and [0068] occupancy grid 52).
21. Regarding Claim 15, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 14, and further, Neitemeier teaches wherein the representation comprises an overlay on an image presented to the operator indicating the relative position of a detected anomaly with respect to the machine (Neitemeier: [0038] and [0068] Note that the colored anomalies are equivalent to an overlay on an image presented to the operator.).
22. Regarding Claim 16, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 1, and further, Neitemeier teaches wherein the one or more operable components comprises a steering system and/or propulsion system of the machine for automating motion of the machine in dependence on the comparison of the reconstructed image with the input image (Neitemeier: [0037], [0038], and [0067] “High obstacles necessitate that the agricultural work machine be steered around them" and "As a result, the detected objects 30, in particular the identified anomalies 19 can be used to control the agricultural work machine 2 and/or its work organs. Thus, the agricultural work machine 2 may be configured to avoid and/or stop at an anomaly 19 autonomously.”).
23. Regarding Claim 17, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 16, and further, Neitemeier teaches to control operation of the propulsion system of the machine to reduce an operating speed of the machine in dependence on the identification of an anomaly within the working environment of the machine (Neitemeier: [0067] “The agricultural work machine 2 may be configured to avoid and/or stop at an anomaly 19 autonomously."). .
24. Regarding Claim 18, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 16, and further, Neitemeier teaches to control operation of the steering system of the machine to guide the machine along an operational path which avoids a location of the detected anomaly within the working environment of the machine (Neitemeier: [0037], [0038], and [0067] “High obstacles necessitate that the agricultural work machine be steered around them" and "As a result, the detected objects 30, in particular the identified anomalies 19 can be used to control the agricultural work machine 2 and/or its work organs. Thus, the agricultural work machine 2 may be configured to avoid and/or stop at an anomaly 19 autonomously.”).
25. Regarding Claim 19, Yamanaka, Parchami, and Neitemeier remain as applied above in Claim 1, and further, Yamanaka teaches an operator assistance system for an agricultural machine, comprising: one or more image sensors (Yamanaka: [0022] on board camera 12);
And the control system of claim 1 (Yamanaka: [0013] on board detection device in vehicles).
26. Regarding Claim 20, Yamanaka teaches a method of monitoring operation of a… machine, comprising: receiving image data indicative of an input image of a working environment of the… machine (Yamanaka: [0013], [0022], and [0040] “on-road obstacle detection devices that detect on-road obstacles in images…”);
Encoding the image data utilizing an encoder network to map the image data to a lower-dimensional feature space (Yamanaka: [0030] Note that the images being processed with an VAE is equivalent to utilizing an encoder network.);
Decoding the encoded data to form a reconstructed image of the working environment (Yamanaka: [0035] Note that the VAE includes an encoded and decoder that reconstructs the image from low dimensional encoding.);
Comparing the reconstructed image with the input image to generate an anomaly map by calculating, pixel-wise, a relative perceptual loss between the input image and the reconstructed image… to the input image and to the reconstructed image (Yamanaka: [0036] and [0040] “The difference between the semantically labelled image and the reconstructed image is computed in the present exemplary embodiment.”);
And controlling operation… in dependence on the comparison (Yamanaka: [0037] and [0038]).
Yamanaka fails to explicitly teach applying a multi-layer artificial neural network to the input image and to the reconstructed image.
However, in the same field of endeavor, Parchami teaches calculating… a relative perceptual loss between the input image and the reconstructed image by applying a multi-layer artificial neural network to the input image and to the reconstructed image (Parchami: [0017] and [0038] “The sparse convolutional neural network can include a plurality of convolutional layers”).
Yamanaka and Parchami are considered to be analogous to the claim invention because they are in the same field of image processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Yamanaka to incorporate the teachings of Parchami to apply a multi-layer artificial neural network to the input image and reconstructed image for comparing the reconstructed image with the input image to generate an anomaly map because it provides the benefit of improving the ability to detect objects while decreasing the time and computing resources to do so. This motivation is explicitly explained in [0013] of Parchami.
Yamanaka and Parchami fail to explicitly teach a method of monitoring operation of an agricultural machine, comprising: controlling operation of one or more operable components associated with the machine in dependence on the comparison.
However, in the same field of endeavor, Neitemeier teaches a method of monitoring operation of an agricultural machine, comprising (Neitemeier: [0002] “This application relates generally to agricultural work machines, and specifically relates to an agricultural work machine…”):
Controlling operation of one or more operable components associated with the machine in dependence on the comparison (Neitemeier: [0036] and [0067] “The at least one further processable image data may be used to control the agricultural work machine and/or its work organs…”).
Yamanaka, Parchami, and Neitemeier are considered to be analogous to the claim invention because they are in the same field of vehicle environment image processing. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to modify Yamanaka and Parchami to incorporate the teachings of Neitemeier to generate and output control signals for controlling the operation of an agricultural machine based on the comparison because it provides the benefit of determining an anomaly in the environment of the work machine as explained in [0017] of Neitemeier. This provides the additional benefit of collision avoidance to increase the safety of the machine, passengers, and surroundings.
Response to Arguments
25. Applicant's arguments filed 6/8/2026 have been fully considered but they are not persuasive.
26. First, the Applicant has alleged "Parchami does not disclose or suggest comparing a reconstructed image with an input image to generate an anomaly map by calculating, pixel-wise, a relative perceptual loss between the input image and the reconstructed image by applying a multi-layer artificial neural network to the input image and to the reconstructed image, as recited in Claim 1." The Examiner disagrees.
In response to the arguments filed 6/8/2026, it appears that the Applicant is arguing the references individually. One cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). As was specifically stated in the Nonfinal Office Action mailed 3/13/2026, Parchami was only used to disclose applying a multi-layer artificial neural network to the input image and to the reconstructed image. Yamanaka was used to teach to compare the reconstructed image with the input image to generate an anomaly map by calculating, pixel-wise, a relative perceptual loss between the input image and reconstructed image. Thus, Parchami still teaches the features for which it was cited.
Additionally, Parchami teaches in [0038] that images in the training dataset are processed using eccentricity analysis and the sparse convolutional neural network. This is equivalent to applying the neural network to the input image(s). Further, Parchami continues to teach in at least [0036] that the sparse convolutional neural network includes multiple layers (502, 504... and 518) to generate an output in order to eliminate all results except detect object classes. The final results (of reducing the resolution of data] being passed to the detection layer of the neural network is equivalent to the applying the neural network to the reconstructed image because only the classes of moving objects. Applying a multi-layer artificial neural network to the input image and reconstructed image provides the benefit of detect moving objects in image data.
As currently claimed, there is no indication on how the multi-layer artificial neural network is applied to the input image and reconstructed image. Therefore, under the broadest reasonable interpretation, inputting image data to the convolutional neural network and passing it through the layers to generate an output is equivalent to applying a neural network to the input image and reconstructed image.
28. Yamanaka (US 20210374440 A1), in view of Parchami (US 20230177842 A1), and in further view of Neitemeier (US 20180084708 A1) teaches all aspects of the invention.
29. Claims 1, 3, and 5-20 remain rejected under their respective grounds and rational as cited above, and as stated in the prior office action which is incorporated herein. Also, although not specifically argued, all remaining claims remain rejected under their respective grounds, rationales, and applicable prior art for these reasons cited above, and those mentioned in the prior office action which is incorporated herein.
Conclusion
30. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
31. Any inquiry concerning this communication or earlier communications from the examiner should be directed to MICHAEL T SILVA whose telephone number is (571)272-6506. The examiner can normally be reached Mon-Tues: 7AM - 4:30PM ET; Wed-Thurs: 7AM-6PM ET; Fri: OFF.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Angela Ortiz can be reached at 571-272-1206. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL T SILVA/Examiner, Art Unit 3663
/ANGELA Y ORTIZ/Supervisory Patent Examiner, Art Unit 3663
