DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
Claims 1-53 are canceled. Claims 54-73 are pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 2/20/24 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
Please note that a thorough review of the reference characters by the Applicant is required. Many of the reference characters have inconsistent definitions in the drawings/specification. There are also multiple reference characters in the drawings not mentioned in the specification (and vice versa).
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “132” has been used to designate both plant positions/locations and stem coordinates; reference character “121” has been used to designated both image/output image and network output/network output array; reference character “601” has been used to designate both hoe trajectory and blade trajectory; reference character “607” has been used to designate both clearance and distance; reference character “269” has been used to designate textured mesh image/textured image, rendered image, re-rendered image, and orthogonal view parameters; reference character “16” has been used to designate both depth information and image/depth image; reference character “111” has been used to designate camera position, camera pose, and input image; reference character “602” has been used to designate forward speed, crop/plant stem, and crop stem location; reference character “18” has been used to designate both per-element depth information and image/depth image. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference characters "602" and "603" have both been used to designate forward speed (see Figs. 70 and 72). Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to because reference character “31” in Fig. 7 should be “34”, reference character “184 (“topRow”)” in Fig. 19 should be “185(“topRow”)”; reference character “255” (by left) should be “254”; the subscripts of the reference characters are difficult to read in Figs. 48 and 49; the reference characters are difficult to read in parts of Fig. 68; the reference characters are difficult to read in part of Figs. 91 and 92. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they include the following reference character(s) not mentioned in the description: “f” and “P” in Fig. 10; S15.2A and S15.2B in Fig. 16; 209 and 210 in Fig. 19; S13.1 in Fig. 20; 215, 220, 221, and D in Fig. 21; 222, 223, 224, and 225 in Fig. 22; 138 and 235 in Fig. 23; v0, v1, v2, and v3 in Fig. 30; d and xm in Fig. 33; L in Fig. 34; 123 in Fig. 38; 138 and 141 in Fig. 45; S46 in Fig. 46; P and Q in Fig. 57; hin in Fig. 58; S60.1 and S60.2 in Fig. 60; p, q, and d(p, q) in Fig. 64; hcamera and hcrop in Fig. 89; 18’’ in Figs. 93 and 96; 803 in Fig. 105; and h in Fig. 107. Corrected drawing sheets in compliance with 37 CFR 1.121(d), or amendment to the specification to add the reference character(s) in the description in compliance with 37 CFR 1.121(b) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(5) because they do not include the following reference sign(s) mentioned in the description: In p. 12, line 9, weeding units 7; in p. 28, lines 7-8 IR sensor intrinsics 96; in p. 44, line 27, image 417; in p. 56, line 9, keypoints 7172; in p. 63, line 28, image 16’; in p. 64, line 17, image 16’’. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The title of the invention is not descriptive. A new title is required that is clearly indicative of the invention to which the claims are directed.
The disclosure is objected to because of the following informalities: In p. 7, lines 16-17, Figures 9A and 9B are not mentioned in the brief description of the drawings. Appropriate correction is required.
Claim Objections
Claim 61 is objected to because of the following informalities: In line 2, “the vicinity” should read –a vicinity–. Appropriate correction is required.
Claim 64 is objected to because of the following informalities: In line 4, “the same stem of the same plant” should read –a same stem of a same plant–. Appropriate correction is required.
Claim 69 is objected to because of the following informalities: In line 3, “memory” should read –and a memory–. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 54-73 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 54 recites the limitation "an image" in line 6. It is unclear and indefinite if this is the same as the “image of a crop” or the “multiple-channel image” previously recited in the claim. Claim 54 also recites the limitation “the stem” in lines 6-7. There is insufficient antecedent basis for this limitation in the claim. Additionally, claim 54 recites the limitation “the provided depth information” in line 10. It is unclear and indefinite if this is referring to the “depth information” or the “per-element depth information” previously recited in the claim.
Claim 55 depends on claim 54 and is therefore rejected under 112(b).
Claim 56 recites the limitation “depth information” in line 2. It is unclear and indefinite if this is the same as the “depth information” or the “per-element depth information” previously recited in the claims.
Claim 57 depends on claim 56 and is therefore rejected under 112(b).
Claims 58-64 depend on claim 54 and are therefore rejected under 112(b).
Claim 65 recites the limitation “amalgamating the coordinates” in line 1. There is insufficient antecedent basis for this limitation in the claim. Claim 64, not claim 54, recites amalgamating coordinates.
Claim 66 recites the limitation “the detected coordinate” in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claims 67-70 depend on claim 54 and are therefore rejected under 112(b).
Claim 71 recites the limitation “the computer system” in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 72 depends on claim 71 and is therefore rejected under 112(b).
Claim 73 depends on claim 70 and is therefore rejected under 112(b).
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 67 and 68 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter as follows.
Claim 67 recites “A computer program, which when executed by at least one processor, performs the method of claim 54” (computer program). Computer programs, per se, do not fall within at least one of the four statutory categories of patent eligible subject matter because a computer program is merely a set of instructions capable of being executed by a computer - the computer program itself is not a process. MPEP § 2106.
A computer program, at best, is a functional descriptive material per se. Descriptive material can be characterized as either "functional descriptive material" or "nonfunctional descriptive material." Both types of "descriptive material" are nonstatutory when claimed as descriptive material per se, 33 F.3d at 1360, 31 USPQ2d at 1759. When functional descriptive material is recorded on some computer-readable medium, it becomes structurally and functionally interrelated to the medium and will be statutory in most cases since use of technology permits the function of the descriptive material to be realized. Compare In re Lowry, 32 F.3d 1579, 1583-84, 32 USPQ2d 1031, 1035 (Fed. Cir. 1994) (discussing patentable weight of data structure limitations in the context of a statutory claim to a data structure stored on a computer readable medium that increases computer efficiency) and >In re Warmerdam, 33 F.3d *>1354, 1360- 61,31 USPQ2d *>1754, 1759 (claim to computer having a specific data structure stored in memory held statutory product-by-process claim) with Warmerdam, 33 F.3d at 1361,31 USPQ2d at 1760 (claim to a data structure per se held nonstatutory). See MPEP 2106.01.
The rejection of claim 67 above may be overcome by amending the claim, for example, to recite: “A non-transitory computer-readable medium storing instructions which, when executed by at least one processor, performs the method of claim 54.”
Additionally, claim 68 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because the broadest reasonable interpretation of the “computer-readable medium” in claim 68 encompasses signals per se. The specification states that a computer-readable medium “may be non-transitory” which clearly includes propagating electromagnetic waves. As understood in light of the specification, the broadest reasonable interpretation of claim 68 encompasses signals which are not within one of the four statutory categories of invention. See MPEP2106.03 I. It is suggested that claim 68 be amended to recite a “non-transitory computer-readable medium” to overcome this rejection.
Allowable Subject Matter
Claims 54-66 and 69-73 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action.
Claims 67 and 68 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 101 and 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph, set forth in this Office action.
The following is a statement of reasons for the indication of allowable subject matter: “Robotic Weed Control Using Automated Weed and Crop Classification” by Wu et al. (hereinafter “Wu”) teaches a weed control system that uses a non-overlapping multi-camera system to track detected weeds (Fig. 1), in which learning-based algorithms are used and 3D locations of the weeds are mapped in real-time (Pg. 2-3 of article, section 1). Wu further teaches using methods in which RGB + NIR data is acquired by a 4-channel JAI camera to provide a pixel-wise semantic segmentation of crops and weeds (Pg. 9 of article, section 4.1; Fig. 6) and generating an inverse depth map and 3D pointcloud of plants (Fig. 7). “Joint Stem Detection and Crop-Weed Classification for Plant-specific Treatment in Precision Farming” by Lottes et al. teaches providing a semantic segmentation of visual input (i.e., RGB or RGB + NIR images) into classes, such as crops, and detecting stem positions for weeds and crops using an FCN network (Pg. 2-3, section III; Fig. 2). However, the cited references, either alone or in combination, do not teach or suggest the following limitations in the ordered combination with the rest of claim 54, such as inter alia, “providing the image to a trained convolutional neural network to generate a response map comprising an image comprising intensity values having respective peaks corresponding to the stem of a plant in the crop; obtaining, from the response map, coordinates corresponding to the respective peaks; and converting the coordinates in image coordinates into stem locations in real-world dimensions using the provided depth information.” Claims 55-73 depend on claim 1 and therefore also contain allowable subject matter.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
“Learning Crop Models for Vision-Based Guidance of Agricultural Robots” by English et al.
“Plant detections and mapping for agricultural robots using a 3D LIDAR sensor” by Weiss et al.
“Plant Localization and Discrimination using 2D+3D Computer Vision for Robotic Intra-row Weed Control” by Gai et al.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Daniella M. DiGuglielmo whose telephone number is (571)272-0183. The examiner can normally be reached Monday - Friday 8:00 AM - 4:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emily Terrell can be reached at (571)270-3717. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Daniella M. DiGuglielmo/Examiner, Art Unit 2666
/EMILY C TERRELL/Supervisory Patent Examiner, Art Unit 2666