DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11/14/2025 has been entered.
Claims 1-5, 8-11, 14-18 and 21-24 are currently pending in U.S. Patent Application No. 18/244,771 and an Office action on the merits follows.
Response to 35 USC § 112 Rejections
In view of the foregoing amendments, previously presented claim rejections under 35 U.S.C. § 112(b) are withdrawn only in part. While claim 4 has been amended to eliminate that language identified as lacking antecedent basis “the first previously identified agricultural object”, claim 3 (second instance at line 3) and 5 (line 2) have not.
Claim Rejections - 35 USC § 101
In view of the foregoing amendments, claim rejections under 35 U.S.C. § 101 are withdrawn. Applicant’s remarks do not address a substantive eligibility analysis at either of Step 2A (to include Prong One and Prong Two), or Step 2B, and instead merely conclude that the longitudinal member as recited serves to realize an improvement broadly, and/or that the amended claims do not ‘recite’ an abstract idea (presumably directed to a Step 2A analysis) in view of the same. The claims as amended however, at least recite limitations/ ‘additional elements’ specific to a particular machine as defined in MPEP 2106.05(b), and such a machine is integral to carrying out the process/method as recited (as distinguished from e.g. ‘apply it’ considerations of MPEP 2106.05(f)). Despite what is arguably commonly implemented boom sprayer structure, analysis at Step 2A does not concern what is ‘WURC’ (see 2106.05(d)). Corresponding rejections to the claims under 35 USC § 101 are withdrawn accordingly
Response to Double Patenting Rejections
Applicant’s remarks request removal of Double Patenting rejections without specifying any manner in which the foregoing amendments may serve to distinguish the claimed invention in a non-obvious manner from claims of reference. The claims as amended now recite that ‘longitudinal’ structure and plurality of treatment units positioned relative thereto (Applicant’s Fig. 23-24), as a system/structural limitations with which a processor configured for vision based targeting of agricultural objects may be implemented, however the newly amended limitations concern structure (e.g. a boom sprayer) that is well-understood/routine and/or conventional in the art, as evidenced by e.g. Zemenchik (US 2019/0311197 A1) (Fig. 1 boom 116), Corti et al. (US 2023/0117884 A1) (Fig. 1, [0069]), Wonderlich et al. (US 2020/0037519 A1) (Fig. 3), Fu et al. (US 2021/0090274 A1) (Fig. 1C-1D), Rees (US 2017/0071188 A1) (Fig. 2A-2C), etc.. After that amendment/claim set dated 8/13/2025 the instant claims differed only in that a pose is determined for orienting the treatment head (now ‘struck’ from the claim(s) – see note below) – which is commonly performed in the art and disclosed in e.g. Robertson as previously applied, Zemenchik, etc.. Claims of reference include that stored imagery corresponding to that of the ‘first pass’ and newly acquired/captured imagery corresponding to that of the ‘second pass’, in addition to “associated real-world geo-spatial locations”. Claims of reference further include determining treatment parameters and that emitting recited. Modification to the claims of reference would be obvious for the reasons presented below, and the claims are rejected under Obviousness type Double Patenting rejections (not patentably distinct over claims of U.S. Patent No. 11,076,589 or U.S. Patent No. 11,751,558) accordingly.
Response to Arguments
Applicant’s arguments with respect to claim(s) as amended 11/14/2025 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument. Remarks suggest Applicant finds Robertson et al (US 2019/0261565 A1) as failing to fairly disclose and/or suggest that longitudinal structure and plurality of treatment units positioned relative thereto. Examiner does note however that ‘longitudinal’, under a plain meaning interpretation (MPEP 2111.01) with an ‘ordinary and customary meaning’ is understood to be a direction along the length of the object/system in question, as distinguished from across it (laterally). In the context of vehicles, this is understood to be from the ‘nose’ to the ‘tail’ – that is the length along a direction in which the vehicle commonly travels/moves. With reference to Applicant’s Fig. 20, this is the x axis associated with vehicle 1610 (and not the y-axis) (Examiner understands along the y-axis of Fig. 20 to be lateral). Applicant’s disclosure does not feature any instance of the term ‘longitudinal’ and as such is not understood to set forth any ‘special definition’ for the term – nor do the remarks referencing support in e.g. Applicant’s Figures 23-24 serve as express intent in the specification, to provide such a special definition that is contrary to a customary meaning. If ‘longitudinal’ is given its customary meaning, Robertson e.g. Fig. 4, may apply, as there is a frame/member/structure that runs longitudinally/length-wise (from nose to tail, front to back given the direction of travel) wherein those robotic arms are illustrated as attached/mounted to the bottom portion of such a member/structure (even if such a structure itself is positioned ‘about’ (see Relative Terminology/ Approximations MPEP 2173.05(b)) the ‘top’ half of the vehicle/system). As illustrated/suggested in Robertson Fig. 4, this permits the surface where the robotic arm would otherwise be mounted, to instead contain (e.g. movable/replaceable) bins/containers in which picked crops/fruit may be placed. For the case of a crop that is e.g. oranges and/or grapes, the agricultural objects/trees/vines grow from a ground of an agricultural environment. Only for the case of those strawberries as disclosed with reference to Fig. 7 and possibly Fig. 6 of Robertson, do the agricultural objects not grow ‘from the ground’ as now required by the claim as amended. These considerations are largely rendered moot in view of e.g. Zemenchik (US 2019/0311197 A1) as applied below. As a point of clarification, Applicant’s remarks at page 10 assert that POSITA would not ‘combine the aerial drone systems’ of Janssen and/or Yang with the system of Robertson, however Yang 108 is not limited to UAV embodiments, and further discloses wheeled robot 108M similar/analogous to e.g. that of Robertson and automated vehicles traversing along lines/rows of agricultural objects/crops (and non-automated if the teaching itself is still analogous under at least a reasonable pertinence theory, if not directed to the same field of endeavor – i.e. vision based selective/targeted crop treatment independent of vehicle type (MPEP 2141.01(a))).
Examiner Note – MPEP 714.03 and 706.07(h)
The claim set filed (11/14/2025) in conjunction with the Request for Continued Examination, does not appear to comply with the manner of making amendments set forth in 37 CFR 1.121. See MPEP § 714.03. More specifically the most recent claim set does not include claim language that was present in the last entered amendment, and does not include the appropriate markings indicating those omitted limitations as being struck from the claim(s). The 06/05/2025 claim set was not entered because it was subject to a Notice regarding Non-Compliant Amendment. The claim set dated 08/13/2025 at approximately line 12, recites “determining a first pose of the real world-agricultural object relative to the treatment unit…”. The claims filed 11/14/2025 do not recite this language, and do not indicate it as being struck from the claim(s) as they appear in the 8/13/2025 amendment. The instant claims also omit “based on the determined first pose, orienting the treatment head to the first real-world agricultural object”, which is required in the 8/13/2025 claim(s) at approximately lines 19-20. Claim 17 is also characterized by these same omissions as identified for the case of claim 1 above. Examiner assumes these deficiencies were not deliberate/intentional, however the 112(b) rejection previously raised for claim 3 and claim 5 also remain unaddressed, and the omission is remarkably similar to the 06/05/205 amendment directing the claim(s) to e.g. a method for irrigation/fertilization/agent application control based on geofencing/geo-spatial location data (e.g. CPC A10C21/005, USPC 700/283 – as distinct from a reference image based object pose determination classified under CPC G06T7/74 and frequently assigned to USPC 382/103 accordingly) . In view of 1) compact prosecution interests, 2) a reduced search burden given the references already made of record, and 3) under an assumption that Applicant’s amendment was bona fide, the 11/14/2025 amendment is being accepted as an adequate reply and considered by the Examiner.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-5, 8-11, 14-18 and 21-24 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Where applicant acts as his or her own lexicographer to specifically define a term of a claim contrary to its ordinary meaning, the written description must clearly redefine the claim term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that claim term. Process Control Corp. v. HydReclaim Corp., 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). The term “longitudinal” in claim(s) 1/17, in view of supporting remarks referencing Applicant’s Figures 23-24, is used in the claim to mean in a direction across, laterally (in that ‘Y’ axis direction associated with 1610, Applicant’s Fig. 20) relative to the vehicle, while the accepted meaning is instead a direction lengthwise, e.g. from nose to tail, front-back and/or along that ‘X’ axis associated with vehicle 1610 (Applicant’s Fig. 20). The term is indefinite because the specification does not clearly redefine the term. See also those remarks presented in the Response to Arguments section above. Applicant may consider amending the claim(s) to read ‘a lateral structure’, and/or provide clarifying remarks.
Claim(s) 1 and 17, also recite that ‘second treatment unit’ as also (in addition to the ‘first’) being associated with ‘a first assembly’ (line 12). It seems likely that the second instance of ‘a first assembly’ is intended to read e.g. ‘a second assembly’ (consistent with that ‘third assembly’ being for the ‘third treatment unit’), however as recited the claim(s) is/are unclear as to if a same assembly is supposed to be that for both the first and second treatment units, or if the first and second treatment units are further defined by/associated with potentially distinct assemblies.
Claim(s) 3 and 5 have not been amended to eliminate those instances of “the first previously identified agricultural object” lacking proper/required antecedent basis – claim 3 at line 3, claim 5 at line 2. See Final mailed 08/26/2025.
Claim 15 recites “orienting the treatment unit to target”, which may be unclear with regards to which of the three recited treatment units is being referenced. For the purposes of compact prosecution Examiner understands it likely that at least that ‘one’ of the treatment units (claim 1 line 36) is referenced. Claim 16 features similar language.
Dependent claims 2-5, 8-11, 14-16, 18 and 21-24 inherit and fail to cure that/those deficiencies identified for the case of independent claim(s) 1/17, and are rejected accordingly.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-5, 8-11, 14-18 and 21-24 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims of U.S. Patent No. 11,076,589 or U.S. Patent No. 11,751,558 and obvious modification thereto similar to the grounds presented in the prior-art based rejections below. Although the claims at issue are not identical, they are not patentably distinct from each other because claims of reference anticipate the claims of the instant application and/or require only minimal/obvious modification to teach/suggest all elements recited in the instant claims. The following additional considerations similarly apply:
• Instant claims and claim(s) of reference recite common subject matter;
• Whereby instant claim(s), recite the open ended transitional phrase “comprising”, and do not preclude those additional elements recited by claims of reference;
• Language/terminology of instant claim(s) constituting minor/slight variations from the claims of reference, if/where present, require interpretations under Broadest Reasonable Interpretation and/or plain meaning definitions (MPEP 2173 and 2111) equivalent to/met by language of the reference claims in view of that corresponding/shared Specification. While the disclosure of reference may not be used as prior art (Double Patenting concerns the claims of reference), portions of the specification which provide support for reference claims may also be examined and considered when addressing the scope of claim(s) of reference and the issue of whether an instant claim defines an obvious variation or falls within the scope of an invention claimed in the claim(s) of reference. See MPEP 804 with reference to In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970).
• Language/terminology of instant claim(s) otherwise not explicitly recited in claim(s) of reference constitute limitations met in view of obvious modification to claims of reference for reasons same/similar to those presented in the prior art based rejections below – It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the claims of reference to further include determining that first pose as recited, and ‘orienting…’ based thereon as taught/suggested by Robertson et al. (US 2019/0261565 A1) (Robertson end effector manipulation in response to object pose relative to the end effector [0109-0110], [0235], etc.,), the motivation(s) being as similarly taught/suggested therein that such a determining and subsequent orientation ensures the corresponding treatment policy is selective in nature and tailored to the object (see Robertson [0285] motivation behind treating select areas, in further view of Janssen [0114] treating objects not previously treated, Zemenchik as applied below, etc.,), in further view of the manner in which such a modification to the claims of reference would serve to increase the marketability and/or adoption of that/those systems/methods in the claim(s) of reference, in a manner characterized by a reasonable expectation of success and without undue experimentation. It would have further been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the claims of reference so as to implement those processing steps in connection with a vehicle/system comprising that ‘longitudinal’ structure/boom comprising a plurality of treatment units as recited, and similarly evidenced as obvious in view of e.g. Zemenchik (US 2019/0311197 A1) (Fig. 1 boom 116), Corti et al. (US 2023/0117884 A1) (Fig. 1, [0069]), Wonderlich et al. (US 2020/0037519 A1) (Fig. 3), Fu et al. (US 2021/0090274 A1) (Fig. 1C-1D), Rees (US 2017/0071188 A1) (Fig. 2A-2C), etc., the motivation as similarly taught/suggested therein and readily recognized by POSITA, such a boom structure enables the system/method to apply treatments with fewer/a reduced number of vehicle passes than would be required otherwise.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
1. Claims 1, 8, 11 and 15-17 are rejected under 35 U.S.C. 103 as being unpatentable over Zemenchik (US 2019/0311197 A1) in view of Robertson et al. (US 2019/0261565 A1) and Yang et al. (US 2020/0401883 A1).
As to claim 1, Zemenchik discloses a method performed comprising:
operating an agricultural treatment system supported or towed by a vehicle having a plurality of wheels (Fig. 1, Fig. 7, etc., agricultural vehicle 102 comprising plurality of wheels 114, and supporting or towing boom 116), the agricultural treatment system comprising:
a longitudinal structure having a top portion and a bottom portion (Fig. 1, 7, 116):
one or more processors comprising hardware (processor 216, [0028], etc.,);
one or more sensors (202);
a first/second/third treatment unit positioned about the bottom portion of the longitudinal structure (first/second/third + versions of applicator 204, [0027], Figures 1, 7, etc., [0025], [0027] “The applicator 204 may be mounted to the boom 116 such that each applicator 204, e.g., applicator 204A, is positioned above a row, e.g., row 128A, for applying an agent to locations of targeted plants associated with the row 128A, when the wheels 114 of the vehicle 102 are positioned between adjacent rows, e.g., rows 128B and 128C, for traveling parallel to the rows 128. However, the scope of the invention covers the applicator 204 mounted between adjacent rows for applying the agent to locations of targeted plants belonging to at least one of the two adjacent rows”), the first/second/third treatment unit comprising a respective treatment head and one or more respective motor(s) each coupled to the first/second/third treatment head via a first/second/third assembly ([0040] “According to one embodiment of the present invention, each applicator 204 includes at least one nozzle 220 coupled to at least one direction means 222. For example, the direction means 222 may be in the form of electric, hydraulic, or pneumatic controlled actuators, including levers, pistons, pins or other known devices which convert a control signal to motion, connected to the nozzle 220 for controlling the direction of the nozzle 220. The control signals may be based upon GPS locations of one or more targeted plants of the plant stand 118, as well as the locations of the applicators 204 relative to the GPS device 206, thereby enabling the direction means 222 to direct the nozzle 220 to point at a GPS location of a targeted plant by rotating and/or translating the nozzle”, [0056]); and
wherein the one more sensors are positioned on the agricultural treatment system to
scan across one or more rows of agricultural objects that grow from a ground of an
agricultural environment (Figures 1, 7, rows 128A-C, [0025-0027], etc.,);
wherein the one or more processors are configured to perform the operations (processor(s) 216, [0028], etc.,) comprising:
while moving the agricultural treatment system along a first pass along a path of the agricultural environment, obtaining a first set of multiple images of the agricultural objects in the agricultural environment (Fig. 6 605 in conjunction with 610 as stored/for reference in future iterations of 605, [0034], [0036] “processor 216 may be configured to generate features of an average plant of the plant stand 118 (e.g., a running average) based upon features determined from previously captured and processed images, and thereafter stored in the memory 218”, [0038], [0043] “In a further embodiment of the present invention, the processor 216 is configured to process the captured images for generating augmented reality (AR) images. Augmented reality is well-known to those of skill in the image processing arts, the details of which will not be discussed here, except to mention that augmented reality includes processing techniques that augment real-world scenes by adding, subtracting, altering, and overlaying features to real-world images. FIG. 5 shows as an AR image 500, according to an embodiment of the present invention. The processor 216 may execute image processing code to augment the image of one or more plants of a group of plants that are growing too close to one another or are morphologically inferior and may augment the image of the plant stand 118 with symbols that represent missing plants (i.e., augment the image of target plants of the plant stand 118 and/or augment the image of the plant stand 118 itself) … For example, the image of the plant 510 is augmented orange with a blue outline, represented by the vertical striping, since it's morphological value, based upon one or more of height, number of leaves, leaf dimensions, or stem diameter, or even based upon a quantization of its overall shape (e.g., does it conform to a standard appearance of a corn plant at a given stage in its development) is below a predefined morphology threshold value”, [0047] “The controller 208 may be configured to store the first and second sets of statistics, as well as AR images, e.g., AR image 500 or spliced-together AR images depicting an agricultural field of which the plant stand 118 is a portion, for future display and/or further analysis”, etc.,);
associating location information with each of the first set of multiple images (GPS 206, [0031] “The image processing code may also include image plant-location code for determining the global coordinates (e.g., a GPS location) of the plants 126 detected in the processed images based further on GPS signals received from the GPS device 206 and a relative location of the corresponding sensor unit 202 with respect to the GPS device 206 (i.e., the coordinates of the sensor unit 202 with respect to the GPS device 206 being located at an origin of a coordinate system)”, [0033], [0037] “216 processes the captured images from the sensor unit 202, the GPS signals received from the GPS device 206, and the locations of the sensor units 202 mounted to the agricultural vehicle 102 relative to the GPS device 206 mounted to the agricultural vehicle 102 for determining a GPS location of each plant captured in the image”, [0038] “the GPS locations of the plants 126 of the captured images of the plant stand 118 are stored in the memory 218, along with the other corresponding characteristics of each of the plants 126, such as morphology values and distances with respect to other plants and/or other objects”, [0051], etc.,);
while moving the agricultural treatment system along a second pass along the path of the agricultural environment, obtaining a second set of multiple images (Fig. 6 605 for any acquisition following a previous one in which similarly acquired images, GPS location information, and related information was acquired and stored);
detecting a first real-world agricultural object from the second set of multiple
images ([0029], [0046] “The first set of statistics may include predicted plant growth rates, predicted soil nutrient levels and predicted plant stand (i.e., crop) yields based upon, for example, the number of plants detected in the plant stand, the morphology of the plants, the distances between plants and plant rows, and/or the global locations of the plants”; Examiner notes Zemenchik further suggests the disclosed invention is aimed at solving deficiencies in the prior art, via the ability to detect and/or quantify characteristics of individual plants – [0004-0005] “What is needed in the art is a system and method for determining characteristics of individual plants, newly emergent and during the growing season, in an automated and efficient manner, then subsequently managing the plant stand, thereby optimizing the NES and harvest for the entire population”);
based on a location of the agricultural treatment system and the associated location information of the first set of multiple images, identifying the detected first real-world agricultural object Fig. 6 610, [0031], [0033] “In one embodiment of the present invention, the characteristics of a plant, as determined by the processor 216, include a morphology value, a position of the plant 126 in relation to positions of other plants 126 in the plant stand 118 (e.g., distances between plants and/or distances of plants to other objects or features, such as plant rows 128), and global coordinates (e.g., a GPS location) of the plant 126 in the plant stand 118. The morphology value may be based on one or more of the following features of a plant, including but not limited to, plant stem size, plant height, number of leaves of the plant, dimensions of one or more of the leaves, and a quantization of the overall shape of the plant 126. In one embodiment, a larger morphology value corresponds to a more mature plant and/or a healthier (i.e., more robust) plant”, [0035-0036], [0043], etc.,; Examiner notes Zemenchik at the minimum suggests referencing stored characteristics determined on the basis of previous passes, to identify potential targets for treatment/agent application broadly, such as instances wherein a plant in question (a potential target) is not characterized by expected/threshold morphological characteristics – in which a ‘new’ plant might fall, and/or a non-desired/pest plant even if characteristics of e.g. a class of weeds for example was previously identified/known); and
Zemenchik further discloses determining treatment parameters associated with the first real-world agricultural object (Fig. 6 615, [0040-0041], [0041] “such as an herbicide directed to kill the targeted plants of the plant stand 118, and the applicator 204 applies a determined dose of plant agent to the GPS locations of the targeted plants in the plant stand 118, as directed by the one or more control signals”); and
based on the determined treatment parameters, emitting a fluid projectile from one of the treatment units, to intercept a first surface portion of the first real-world agricultural object (Fig. 6 620, [0042] “In one embodiment of the invention, the plant agent is an herbicide designated to kill targeted plants, such as, for example, plants of the plant stand 118 that are growing too close together, or growing too far away from a plant row, or are morphologically inferior (e.g., below a predefined morphology standard for plants as input by an operator, or below a standard deviation of morphology values of plants previously imaged in the plant stand 118). However, the scope of the present invention covers all types of agents that have effects on plants and/or the plant stand, or on weeds of the plant stand, including but not limited to, pesticides, insecticides, fungicides, fertilizers, soil treatment agents, such as nitrogen, and even water”, etc.,).
Robertson evidences the obvious nature of a method performed by an agricultural treatment system comprising one or more processors comprising hardware (Fig. 1, [0082-0091]), one or more sensors (camera(s), [0118], [0132-0133], [0173], additional sensors [0098]), and a treatment unit, the one or more processors configured to perform operations ([0275] “The picking robot can be equipped to perform several functions in addition to picking, including the ability to spray weeds or pests with suitable herbicides and pesticides, or to reposition trusses (i.e. stalk structures) to facilitate vigorous fruit growth or subsequent picking”, [0554] “A robotic fruit picking system in which the computer vision based subsystem locates specific parts of a plant or specific plants that require a targeted localized application of chemicals such as herbicides or pesticides”) comprising:
while moving the agricultural treatment system along a first pass along a path of an agricultural environment, obtaining a first set of multiple images of agricultural objects in the agricultural environment ([0136-0137] “Under the control of the Robot Control Subsystem, the camera captures images of the scene from multiple viewpoints”, [0141], [0268], [0354], etc.,);
associating location information with each of the first set of multiple images ([0268] “Because picking robots maintain a continuous estimate of their position in a map coordinate system, they can gather geo-referenced data about the environment. A useful innovation is therefore to have robots log undesirable conditions that might require subsequent human intervention along with a map coordinate and possibly a photograph of the scene”);
while moving the agricultural treatment system along a second pass along the path of the agricultural environment, obtaining a second set of multiple images (see that capturing disclosure as identified above for the case of an initial pass);
detecting a first real-world agricultural object from the second set of multiple images ([0137], [0145] “Target detection. Target fruit is detected automatically in images obtained by a camera mounted to the Picking Arm or elsewhere. A machine learning approach is used to train a detection algorithm to identify fruit in RGB colour images (and/or in depth images obtained by dense stereo or otherwise)”);
determining a first pose of the first real-world agricultural object relative to a treatment unit positioned on the agricultural treatment system ([0133] “The camera attached to the robot arm is moved under computer control to facilitate the detection of target fruits, estimation of their pose (i.e. position and orientation), and determination of their likely suitability for picking. Pose estimates and picking suitability indicators associated with each target fruit may be refined progressively as the arm moves”, [0134], [0138-0139] “3. Approximate estimates of pose and shape are recovered for each detected fruit. [0139] 4. More accurate estimates of pose and shape are recovered by combining information in multiple views with statistical prior knowledge. This is achieved by adapting the parameters of a generative model of strawberry appearance to maximize agreement between the predictions and the images”, [0148], [0149] “The robot determines the pose of target fruit using images obtained from multiple viewpoints, e.g. using a stereo camera or a monocular camera”, [0160] (i) the estimated pose and shape of the fruit and its stalk, [0150], [0152], [0156], [0162], etc.,), the treatment unit comprising a treatment head (see Robertson disclosure of ‘picking head’, [0275], etc., in view of [0285] “By using computer vision to locate specific parts of the plant or instances of specific kinds of pathogen (insects, dry rot, wet rot, etc.), robots can apply such chemicals only where they are needed, which may be advantageous (in terms of cost, pollution, etc.) compared to treatment systems that require spraying the whole crop. To facilitate doing this kind of work using a robot that is otherwise used for picking it would obviously be advantageous for the robot to support interchangeable end effectors, including one that can be used for picking and one that can be used for spraying”);
based on a location of the agricultural treatment system and the associated location information of the first set of multiple images, identifying the detected first real-world agricultural object as a new agricultural object or a previously identified agricultural object ([0553] “the computer vision based subsystem is used to classify a fruit, and in which the system allows a grower to adjust thresholds for classifying the fruit”, [0077], [0146-0147] “A decision forest classifier or convolutional neural network (CNN) may be trained to perform semantic segmentation, i.e. to label pixels corresponding to ripe fruit, unripe fruit, and other objects. Pixelwise labelling may be noisy, and evidence may be aggregated across multiple pixels by using a clustering algorithm. [0147] 2. A CNN can be trained to distinguish image patches that contain a target fruit at their centre from image patches that do not. A sliding window approach may be used to determine the positions of all image patches likely to contain target fruits. Alternatively, the semantic labelling algorithm 1 may be used to identify the likely image locations of target fruits for subsequent more accurate classification by a (typically more computationally expensive) CNN”, [0158], [0164], [0271-0274] “2. A useful and related idea is to have picking robots store the map coordinate system locations of all detected fruit (whether ripe or unripe) in computer memory. This makes possible several innovations: … [0274] … to store the map coordinate system position of unripe fruits that have been detected but not picked in computer memory so that a robot can locate not-yet-picked target fruits more quickly on a subsequent traversal of the crop row. In a simple embodiment, the position of previously detected but not yet picked fruits is stored so the robot can return directly to the same position on the subsequent traversal without spending time searching”; Examiner notes a plain meaning reading of ‘new agricultural object’ may include one that was not previously detected and/or detected as e.g. unripe and stored in the disclosed map coordinate system whereas a ‘previously identified agricultural object’ may have been and/or otherwise satisfies classification constraints identifying it as e.g. ‘ripe’/suitable for picking, treatment, etc., wherein Robertson discloses both in view of [0274-0275] and various CV based classification related thereto – see remarks above); and
determining treatment parameters associated with the first real-world agricultural object ([0275] “The picking robot can be equipped to perform several functions in addition to picking, including the ability to spray weeds or pests with suitable herbicides and pesticides, or to reposition trusses (i.e. stalk structures) to facilitate vigorous fruit growth or subsequent picking”, [0285], [0458], [0554] “locates specific parts of a plant or specific plants that require a targeted localized application of chemicals such as herbicides or pesticides”); As identified above, the recited language does not serve to specifically link any treatment parameters with that identifying of the object as being a class of either ‘new’ or ‘previously identified’, and Robertson is explicit in disclosing that ‘specific parts of a plant or specific plants’ may be treated with targeted localized application of chemicals, in addition to that map information for previously detected agricultural objects ([0274]) facilitating several functions to include e.g. treatment ([0275]);
based on the determined first pose, orienting the treatment head to the first real-world agricultural object (Robertson end effector manipulation in response to object pose relative to the end effector [0109-0110], [0235], etc.,); and
based on the determined treatment parameters, emitting a fluid projectile from one of the treatment heads to intercept a first surface portion of the first real-world agricultural object (Robertson [0275], [0285] “By using computer vision to locate specific parts of the plant or instances of specific kinds of pathogen (insects, dry rot, wet rot, etc.), robots can apply such chemicals only where they are needed, which may be advantageous (in terms of cost, pollution, etc.) compared to treatment systems that require spraying the whole crop”, [0285] “To facilitate doing this kind of work using a robot that is otherwise used for picking it would obviously be advantageous for the robot to support interchangeable end effectors, including one that can be used for picking and one that can be used for spraying”, [0458] “including the ability to spray weeds or pests with suitable herbicides and pesticides, or to reposition or prune trusses to facilitate vigorous fruit growth or subsequent picking”, [0554] “targeted localized application of chemicals such as herbicides or pesticides”; wherein the disclosed types are at least a treatment for insects – e.g. insecticide, as distinguished from dry rot e.g. herbicide/fungicide; [0557], etc.,).
Yang further evidences the obvious nature of determining a treatment policy on the basis of previously identified information pertaining to individually recognized agricultural objects (Fig. 6 612 in view of that stored history of Fig. 6 608, [0002] “Numerous different technologies exist for identifying a type of a plant (i.e., classify the plant) based on image data, but these technologies are not concerned with recognizing individual plants as distinct from other individual plants. However, the ability to recognize and distinguish between individual plants may be useful for a variety of purposes. For example, if individual plants can be identified and/or distinguished from each other over time, it may be possible to track individual plants' growth (or lack thereof), disease progression, fruit development (or lack thereof), and so forth. Tracking these metrics enables other applications, such as distinguishing portions of crop fields that are not meeting growth expectations (e.g., due to disease, insufficient or too much irrigation, insufficient or too much fertilizer, etc.) from other portions that are meeting growth expectations”, [0004] “Being able to recognize individual plants, as opposed to only classifying each plant as a particular type of plant, may enable more fine-tuned agricultural management”, [0046], [0052], [0059], etc., see also [0005] “a unique identifier may be generated for each unique plant. A plurality of plants, e.g., of a field or farm, may be indexed in a database using their respective unique identifiers”, [0048], wherein stored/associated data comprises that ‘additional attribute data’ (comprising pose/location and spatial dimension information) from previous/historical identification instances, claim 6, and [0066-0067]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to modify the system and method of Zemenchik such that the plant detection disclosed therein and subsequent characteristic identification further comprises identifying the detected first real-world agricultural object as a previously identified agricultural object (associated with a specific identity, under a narrower interpretation than otherwise afforded to ‘previously identified’ in the context of a plant that is ‘recognized’ broadly) as taught/suggested by Robertson and Yang, the motivation being, as similarly taught/suggested therein, that such a determination/identifying would ensure the corresponding treatment is selective in nature and tailored to a target object and/or specific needs of a unique plant/object (see Robertson [0285] motivation behind treating select areas/crops on the basis of previous harvest/treatment/intervention, Yang [0069-0070], etc.,).
As to claim 8, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 1.
Zemenchik in view of Robertson and Yang teaches/suggests the method further comprising accessing a plurality of images depicting a plurality of previously identified agricultural objects including the previously identified agricultural object (Zemenchik [0028], [0060] “216 executes a two-step imaging process, generating first processed images based upon the captured images, as illustrated in FIG. 4B, from which the one or more characteristics of each plant 126 may be determined, and then generating second processed images ( e.g., the AR images) based on the first processed images, which are particularly useful in presenting time-lapsed displays of the plant stand 118, based upon images captured at different times during the growing season. In this manner, an operator may determine and illustrate (via the display 230, for example) the effectiveness of applying, via the applicator 204, different agents 224 on the development of the plant stand 118 and on the changes to the predicted yield as a result of applying the agent 224 as compared to the predicted yield if the agent 224 had not been applied”; see also Robertson disclosure as identified for the case of claim 3 (below) in further view of the manner in which corresponding training images are class/type specific – in other words the disclosed plurality of previously identified agricultural objects includes strawberries, raspberries, pears, apples, etc., and the/a previously identified object may be any of these predefined classes else ‘new’/unrecognized; See also Yang [0067], [0070] “For example, a user could provide the search query "show me plants infested with mites" at a GUI similar to 550. The search query may be provided to agriculture knowledge system 102. In response, agriculture knowledge system 102 may search database for plants in a particular field known to be infested with mites. The results may include a list of individual plants that were observed, e.g., in their respective time-sequence of images, to be infested with mites. A user may select any one of these results to view statistics about the plant associated with the selected result, to view a time-sequence of digital images of the plant associated with the selected result, etc.”, under an alternate plain meaning interpretation).
As to claim 11, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 1.
Zemenchik in view of Robertson and Yang teaches/suggests the method further comprising determining a first pose of the agricultural treatment system (Zemenchik as identified above, in view of GPS 206 associated with vehicle/tractor, SLAM disclosure [0030], etc.; Robertson [0098-0099] “The purpose of the pose determination component is to allow the robot to determine its current position and orientation in a map coordinate system for input to the Control Component. Coarse position estimates may be obtained using differential GPS but these are insufficiently accurate for following rows of crops without collision. Therefore, a combination of additional sensors is used for more precise determination of heading along the row and lateral distance from the row”, [0103], [0221-0222], [0225], etc.,).
As to claim 15, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 1.
Zemenchik in view of Robertson and Yang teaches/suggests the method further comprising orienting the treatment unit to target the first real-world agricultural object (Zemenchik [0040-0041], [0056], etc.; Robertson [0068], [0092] “The Total Positioning System and the Picking Arm operate under the control of the Control Subsystem, which uses input from the Computer Vision Subsystem”, [0103], [0109-0110], etc., disclosing end effector pose control as applied to effector embodiments comprising implements for delivering/spraying herbicide/insecticide in conjunction with orienting the robot as a whole [0103]).
As to claim 16, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 15.
Zemenchik in view of Robertson and Yang teaches/suggests the method wherein orienting the treatment unit comprises moving a treatment head of the treatment unit about a first axis and a second axis (Zemenchik [0040-0041], [0056], etc.; Robertson [0109] “The Picking Arm is a robot arm with several (typically 6) degrees of freedom that is mounted to the main body of the robot. Whereas the purpose of the Total Positioning System is to move the whole robot along the ground, the purpose of the Picking Arm is to move the Picking Head (and its computer vision camera) to appropriate positions for locating, localizing, and picking target fruit”).
As to claim 17, this claim is the system claim corresponding to the method of claim 1 and is rejected accordingly.
2. Claims 2-5 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zemenchik (US 2019/0311197 A1) in view of Robertson et al. (US 2019/0261565 A1), Yang et al. (US 2020/0401883 A1) and Redden et al. (US 2015/0015697 A1).
As to claim 2, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 1.
Zemenchik discloses feature extraction/known digital image feature detection means of e.g. [0029-0030], but fails to explicitly disclose feature matching objects in the second set of multiple images to the first set of multiple images to determine that a prior image of the first set is similar to a captured image of the second set.
Robertson evidences the obvious nature of feature matching objects in the second set of multiple images to the first set of multiple images to determine that a prior image of the first set is similar to a captured image of the second set (Robertson discloses accessing as at least training image data of various target fruit ([0150], [0567], [0586-0587], claim 27 “from a database of existing images with associated expert derived ground truth labels”, etc.,) and Robertson discloses various previously identified agricultural object classes ([0061] “The picking system is applicable to a variety of different crops that grow on plants (like strawberries, tomatoes), bushes (like raspberries, blueberries, grapes), and trees (like apples, pears, loganberries). In this document, the term fruit shall include the edible and palatable part of all fruits, vegetables, and other kinds of produce that are picked from plants (including e.g. nuts, seeds, vegetables) and plant shall mean all kinds of fruit producing crop (including plants, bushes, trees). For fruits that grow in clusters or bunches (e.g. grapes, blueberries), fruit may refer to the individual fruit or the whole cluster”) to include but not limited to e.g. strawberries, raspberries, tomatoes, apples, plums, peaches, pears, cherries, olives, their corresponding/associated trees/bushes, etc., ([0427], [0614]) among other broader objects ([0409]) - POSITA would further recognize the manner in which imagery acquired via previous/initial passes may serve as training samples).
Redden further evidences the obvious nature of feature matching between stored/reference imagery and newly acquired/second images to determine that a newly acquired image corresponds to one or more of those reference/first images ([0108] “the plant portion can be modeled using morphological images from a limited number of viewing angles. In another variation of the method, the obstructed plant portions are modeled by identifying a reference point (e.g., a plant feature, such as a stem, node, or leaf) from the morphological measurement, determining a location of the plant portion of interest relative to the reference point (e.g., retrieving the location from a historical measurement or model), generating a new virtual model of the plant portion including the reference point based on the new measurements, comparing the new virtual model to the historical model by aligning or otherwise matching the reference points, and associating the closest matching plant features in the new model with the plant features in the historical model (e.g., wherein the plant features can be corrected for estimated plant growth). In another variation of the method, an obstructed plant can be identified by identifying a reference plant within the new morphological measurement, determining a location of the reference plant within a historical model, determining a location of the plant relative to the reference plant within the historical model, generating a new virtual model of the plants within the geographical area including the reference plant based on the new measurements, comparing the new virtual model to the historical model by aligning or otherwise matching the reference plants, and associating the closest matching plants in the new model with the plants in the historical model (e.g., based on plant growth pattern, relative size, relative shape, relative position, relative orientation, etc.). However, obstructed plants or plant features can be otherwise identified” in further view of [0046] “The plant identifier can be a geographic location”, [0069], [0074] “treatment history, such as treatment history for an individual plant and treatment history for a plant field portion … The data for each plant, genotype, or field portion is preferably associated with a location”, [0088] “Individual plants are preferably identified by the plant location in successive data collection sessions, as visual, tactile, and other physical characteristics of plants tend to vary over time. However, individual plants can be uniquely identified based on a characteristic that has a low variation rate, such as stalk or trunk markings. Alternatively, the locations of each individual plant can be premapped, wherein the plant characteristics extracted from each image is correlated to the pre-mapped plant based on the data location”, [0099]), in addition to treatment parameters determined on the basis of plant identity and associated characteristics ([0041] “The plant data (e.g., plant indices) can additionally or alternatively be used to determine treatment parameters for one or more plants, such as which plants to treat, when to treat the plants, which treatment method to use, where on each plant to treat, how much of a treatment to use (e.g., what concentration of fertilizer should be used), or any other suitable treatment parameter. The treatments can include growth promotion, growth retardation, necrosis, or any other suitable treatment”, [0044] “or individual treatments (e.g., individual plants within the same row or plant field can be treated with different treatments, wherein the individual treatments can be manually or automatically applied by the system)”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Zemenchik in view of Robertson and Yang to comprise a feature matching as disclosed in Redden, the motivation as similarly taught/suggested therein that such a feature matching may facilitate individual plant identification in a manner accounting for occlusion and changes/growth in the agricultural object over time, complementary to an identification on the basis of location information as disclosed in Zemenchik and Robertson.
As to claim 3, Zemenchik in view of Robertson, Yang and Redden teaches/suggests the method of claim 2.
Zemenchik in view of Robertson, Yang and Redden teaches/suggests the method wherein the previously identified agricultural object comprises a portion of a two-dimensional (2D) or three-dimensional (3D) image of the Zemenchik stored imagery 2D imagery, and augmented imagery based thereon, of 218, see mapping/disclosure identified above, in further view of 3D image embodiments suggested in e.g. [0045] “such as augmenting the images of the plant stand 118 with 2D maps of colored polygons or pixels, or projecting 3D features onto the image of the 2D plant stand 118”; Robertson [0148] “Pose may be modelled as a 4-by-4 homography that maps homogenous 3D points in a suitable fruit coordinate system into the world coordinate system. The fruit coordinate system can be aligned with fruits of specific types as convenient. For example, the origin of the coordinate system may be located at the point of intersection of the body of the fruit and its stalk and the first axis points in the direction of the stalk. Many types of fruit (such as strawberries and apples) and most kinds of stalk have a shape with an axis of approximate rotational symmetry. This means that 5 degrees of freedom typically provide a sufficiently complete representation of pose for picking purposes, i.e. the second and third axes of the fruit coordinate system can be oriented arbitrarily”, [0150] “A useful innovation is to use a learned regression function to map images of target fruits directly to their orientation in a camera coordinate system. This can be achieved using a machine learning approach whereby a suitable regression model is trained to predict the two angles describing the orientation of an approximately rotationally symmetric fruit from images (including monocular, stereo, and depth images). This approach is effective for fruits such as strawberries that have surface texture that is aligned with the dominant axis of the fruit. Suitable training images may be obtained using a camera mounted to the end of a robot arm”, [0151-0152], etc., identifying how certain pre-identified objects may comprise features readily indicative of a dominant axis/orientation, in addition to that important and trained/learned relationship between a pose estimate for a target fruit and information regarding where the stalk may then attach/be located; re. both 2D and 3D image/data embodiments see Robertson [0154-0156]; see also e.g. Redden [0054], [0060], [0062]; while not relied upon Gurzoni (see below) is also of note).
As to claim 4, Zemenchik in view of Robertson, Yang and Redden teaches/suggests the method of claim 3.
Zemenchik in view of Robertson, Yang and Redden further teaches/suggests the method wherein identifying the detected first real-world agricultural object as the previously identified agricultural object comprises comparing a classification of theZemenchik as identified above under an interpretation that a ‘classification’ broadly may be one related to e.g. the plant falling within and/or below a morphology threshold – e.g. as described in [0041-0042] small, not healthy, inferior etc., and/or non-desirably located/positioned, etc.,; Robertson [0146-0147] “A decision forest classifier or convolutional neural network (CNN) may be trained to perform semantic segmentation, i.e. to label pixels corresponding to ripe fruit, unripe fruit, and other objects. Pixelwise labelling may be noisy, and evidence may be aggregated across multiple pixels by using a clustering algorithm. [0147] 2. A CNN can be trained to distinguish image patches that contain a target fruit at their centre from image patches that do not. A sliding window approach may be used to determine the positions of all image patches likely to contain target fruits. Alternatively, the semantic labelling algorithm 1 may be used to identify the likely image locations of target fruits for subsequent more accurate classification by a (typically more computationally expensive) CNN”).
As to claim 5, Zemenchik in view of Robertson, Yang and Redden teaches/suggests the method of claim 4.
Zemenchik fails to explicitly disclose the method further comprising determining that the Zemenchik at least suggests motivation for such a same object determination in e.g. [0004-0005], and Robertson more explicitly provides motivation for such a same object determination in e.g. [0271], [0274], etc.,.
Yang as applied in the rejection of claim 1 further evidences the obvious nature of determining that the previously identified agricultural object and the detected first real-world agricultural object are the same object in the agricultural environment ([0002], [0004] “Being able to recognize individual plants, as opposed to only classifying each plant as a particular type of plant, may enable more fine-tuned agricultural management”, [0046], [0052], [0059] “that the two images depict the same plant”, etc.,).
Redden further evidences the obvious nature of determining that the previously identified agricultural object and a detected first real-world agricultural object are the same object ([0108], see disclosure identified above for the case of claim 2 – as such an identification enables individualized treatment as disclosed in Redden).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Zemenchik as proposed, to comprise determining that the previously identified agricultural object and the detected first real-world agricultural object are the same object as taught/suggested by Yang, the motivation as similarly taught/suggested therein that such a same object determination enables the system/method to determine effectiveness of any past/related treatment(s) in addition to prognosis/predictions for an individual plant on the basis of historical data (e.g. [0060] of Zemenchik with respect to the effectiveness of previous/historical applications more broadly, but for the case of one or more individual plant(s) in further view of that proposed modification for the case of claim 1 above).
As to claim 18, this claim is the system claim corresponding to the method of claim 2 and is rejected accordingly.
3. Claims 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Zemenchik (US 2019/0311197 A1) in view of Robertson et al. (US 2019/0261565 A1), Yang et al. (US 2020/0401883 A1) and Alexander et al. (US 2021/0007287 A1).
As to claim 9, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 8.
Zemenchik in view of Robertson and Yang fails to explicitly disclose the method wherein identifying the detected first real-world agricultural object as the previously identified agricultural object comprises performing template matching of one or more portions of captured sensor data depicting the first real-world agricultural object and one or more portions of the plurality of images depicting the plurality of preidentified agricultural objects.
Alexander however evidences the obvious nature of template matching for agricultural object classification, between one or more portions of captured sensor data depicting an object and one or more portions of a plurality of images depicting a plurality of preidentified agricultural objects ([0066] “In one example, the system implements template matching or object recognition techniques to identify leaves in the cluster of pixels associated with the first plant in the module-level image, such as based on template images or models of leaves of plants of the same type and at similar growth stages as the first set of plants in the first module”, [0094], [0153] “For example, the system can: retrieve a first image of a new plant at a particular stage of development (e.g., at a particular time after seeding); access a set of template images of previous plants at or near this same stage of development and of known final outcomes; and then implement computer vision techniques to match the first image to a nearest second image in the set of template images. If the known outcome of the second plant is positive (e.g., near an "ideal" plant), the system can continue to implement the existing grow schedule for the new plant. However, if the known outcome of the second plant is negative or far from an "ideal" final outcome, the system can: access a subset of template images to include only images of previous plants at or near this same stage of development and of known positive final outcomes; implement computer vision techniques to match the first image to a nearest third image in this subset of template images”).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Zemenchik as proposed so as to comprise implementing template matching between sub-images/portions of the captured images and one or more sub-images/portions of a plurality of images depicting a plurality of preidentified agricultural objects as taught/suggested by Alexander, the motivation as similarly taught/suggested therein that such a template matching may serve as an efficient means of classification, identification, etc., minimally requiring analysis of only those sub-images/portions (e.g. leaves as distinguished from the plant as a whole) of interest/importance concerning one or more features of interest (e.g. for an instance that leaves (or leaves in a particular portion of the plant – e.g. new growth, oldest growth, etc.,) as an example are the most indicative of nutrient deficiency, pest damage, etc., as suggested by Alexander).
As to claim 10, this claim closely corresponds to claim 9 with the exception that sub-images/portions/crops are derived from Examiner has previously taken Official Notice to common forms of pre-processing including noise removal, contrast enhancement, image resizing and color correction – as applicable to that ‘a first image’.
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Zemenchik as proposed such that captured image sensor data is subject to pre-processing as identified above, in deriving ‘a first image’ and/or otherwise, the motivation being that such a pre-processing (and resultant sub-images accordingly) may serve to facilitate more accurate and/or efficient subsequent processing based thereon (e.g. resizing for input into a network/model, noise removal/contrast enhancement to account for variations in illumination, cropping to minimize features for comparison (e.g. eliminating background area that may otherwise contribute to false positive matches), etc.,) in further view of a sub-image extraction/ cropping as taught by Alexander and for the reasons identified above in the rejection of claim 9.
4. Claim 14 is rejected under 35 U.S.C. 103 as being unpatentable over Zemenchik (US 2019/0311197 A1) in view of Robertson et al. (US 2019/0261565 A1), Yang et al. (US 2020/0401883 A1) and Janssen et al. (US 2022/0254155 A1).
As to claim 14, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 1.
Zemenchik in view of Robertson and Yang teaches/suggests the method wherein the treatment parameters is based on a treatment history associated with the previously identified agricultural object (Zemenchik [0060-0061], Yang further suggests the manner in which individual historical information may serve in decision making – in the event that one or more treatments to date are ineffective and/or sub-optimal; Robertson further discloses reinforcement learning in a broader context – e.g. [0247-0249], [0421] disclosing less optimal strategies for picking and/or control may be updated to become more effective – applicable to a treatment similar to a picking).
Janssen further evidences the obvious nature of treatment parameters based on a treatment history associated with the previously identified agricultural object (Janssen [0114] “Alternatively, the treatment arrangement 270 of the treatment device 200 is provided with a supplied map, indicating how the field has been treated in the past, and treats the plantation in the field based on the supplied map”, [0044]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Zemenchik as proposed such that one or more treatment parameters account for a treatment history as taught/suggested by Robertson, Janssen and/or Yang, the motivation as similarly taught/suggested therein and readily recognized by PHOSITA that such a treatment may ensure that ineffective treatments are not wastefully repeated, are updated/optimized as desired to be more effective, and/or that a decision to forego treatment entirely may be made if e.g. plant prognosis is negative.
5. Claims 21 and 23 are rejected under 35 U.S.C. 103 as being unpatentable over Zemenchik (US 2019/0311197 A1) in view of Robertson et al. (US 2019/0261565 A1), Yang et al. (US 2020/0401883 A1) and Pilney et al. (US 2016/0316735 A1).
As to claim 21, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 1.
Zemenchik in view of Robertson and Yang teaches/suggests the method wherein the longitudinal structure comprises two wheels attached to the bottom portion of the longitudinal structure (Zemenchik Fig 2 illustrates wheel 114 in close proximity to boom 116, however does not clearly/explicitly identify the wheel illustrated as being attached to the bottom portion of 116, as opposed to attached to the tractor/vehicle).
Pilney evidences the obvious nature of two or more wheels attached to the bottom portion of a structure/boom carrying sprayer/treatment hardware (Fig. 1-3, wheels 5, Abs “A deflectable touchdown wheel system for a sprayer boom of an agricultural sprayer is provided that includes a touchdown wheel that deflects transversely when contacting the ground at an angle, damping ground contact impact forces at the sprayer boom reducing reactionary forces such as external steering inputs at the sprayer vehicle”, etc.,).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Zemenchik as proposed, so as to further comprise two wheels attached to the bottom portion of the longitudinal structure as taught/suggested by Pilney, the motivation as similarly disclosed therein that such a wheel may serve to protect the boom/associated sensors/hardware during operation.
As to claim 23, this claim is the system claim corresponding to the method of claim 21 and is rejected accordingly.
6. Claims 22 and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Zemenchik (US 2019/0311197 A1) in view of Robertson et al. (US 2019/0261565 A1), Yang et al. (US 2020/0401883 A1) and Wonderlich et al. (US 2020/0037519 A1).
As to claim 22, Zemenchik in view of Robertson and Yang teaches/suggests the method of claim 1.
Zemenchik in view of Robertson and Yang teaches/suggests the method wherein the vehicle comprises a tractor configured to support the agricultural treatment system (Zemenchik tractor embodiment for vehicle 102 [0021], in further view of embodiments wherein the boom is located proximate to the ‘back portion’ of the vehicle 108, as distinguished from the front 106).
Zemenchik fails to explicitly disclose towing.
Wonderlich evidences the obvious nature of towing an agricultural treatment system (Fig. 1, [0020]).
It would have been obvious to a person of ordinary skill in the art, before the effective filing date, to further modify the system and method of Zemenchik as proposed, so as to further comprise towing an agricultural treatment system as taught/suggested by Wonderlich, for that tractor as disclosed in Zemenchik (and also in Wonderlich), the motivation being as readily recognized by POSITA that such a towed/detachable system may enable its use in conjunction with a tractor not limited to solely treatment/operations with one of such system/attachments – e.g. enabling substitution with a soil aerator, planter, mowing deck, etc., useful prior to and after a growing season but for different and/or related tasks.
As to claim 24, this claim is the system claim corresponding to the method of claim 22 and is rejected accordingly.
Additional References
Previously identified Gurzoni, Jr et al. (US 10,891,482 B2) further evidences the obvious nature of associating geo-referencing data with captured crop/agricultural object imagery and a plurality of analysis embodiments (e.g. fruit and flower disease estimation), comparable to those of Robertson made efficient by being “limited to areas of the 3D image where fruits and/or flowers are to be considered” (Gurzoni col 14 lines 1-15). Redden also evidences the obvious nature of identifying individual plants on the basis of stored location information ([0088] “Individual plants are preferably identified by the plant location in successive data collection sessions, as visual, tactile, and other physical characteristics of plants tend to vary over time. However, individual plants can be uniquely identified based on a characteristic that has a low variation rate, such as stalk or trunk markings. Alternatively, the locations of each individual plant can be pre-mapped, wherein the plant characteristics extracted from each image is correlated to the pre-mapped plant based on the data location”). Robertson teaches generally, in e.g. [0274] “Therefore, a useful innovation is to store the map coordinate system position of unripe fruits that have been detected but not picked in computer memory so that a robot can locate not-yet-picked target fruits more quickly on a subsequent traversal of the crop row” and alternative to picking, that spraying of [0275]. Upon subsequent detection, Robertson suggests ‘new’ object/fruit equivalents that are e.g. those now detected as sufficiently ripe. Teachings for embodiments of Robertson directed to fruit/objects yet-to-be-picked are analogous to/applicable to yet-to-be-treated objects (see [0285]). Robertson further relies upon that stored/reference geo-referenced location information to efficiently navigate to such objects, in addition to planning/dispatching the system to locations where future harvesting/treating operations would be of most economic (see yield mapping embodiments [0272], route planning [0207], [0492], etc.,). See also Robertson at [0268-0273] “A useful and related idea is to have picking robots store the map coordinate system locations of all detected fruit (whether ripe or unripe) in computer memory. This makes possible several innovations: …”.
Inquiry
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IAN L LEMIEUX whose telephone number is (571)270-5796. The examiner can normally be reached Mon - Fri 9:00 - 6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at 571-272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IAN L LEMIEUX/
Primary Examiner, Art Unit 2669