Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant generally discusses the prior action at 6 of the response. This is neither a reasoned traverse of the prior rejections nor a petition. The prior action apprises applicant of such information and references as may be useful in judging of the propriety of continuing the prosecution of his application. See 35 U.S.C. 132. It further identifies the pertinent portions of the references in a manner appropriate for applicant to respond to the rejection. Applicant does not identify elements of the claims that are not apparent from the references and the office action, read as a whole; and Applicant is not proceeding pro se.
Applicant notes at 6 of the response that the claims have been amended to address the 112(b) rejections of Claims 8-9, 11-14 and 20. In light of the amendments, those rejections are withdrawn.
Claim Objections
Claim 1 is objected to because of the following informalities: Claim 1, Line 6, “a plurality positioning members” should read “a plurality of positioning members”.
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1 (and dependents therefrom as a result), 3, 5 (and dependents therefrom as a result), 12, and 20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites that “a number of ears present on a plant, a height of an ear from the ground, a health of the ears, and an angle of a stalk in relation to a ground surface” are determined based on crop characteristics. However, in the specification as filed, these are the crop characteristics. See e.g. page 3 lines 27-32.
Claim 3 has been amended to require all of a camera, a sensor, a transceiver, and a stereo sensor. In the specification they appear to be alternatives, particularly the sensor and the stereo sensor.
Claim 5 recites “a plant productivity”. This is not described in the specification. Page 5 of the specification mentions “an ear productivity”, which does not appear to be the same thing.
Claim 12 has been amended to include limitations specifying a meaning of optimal; however, it does not appear that the specification as filed includes a description that the optimal placement is “wherein the optimal placement is the position at which an ear of a plant is most visible”. Optimal appears twice in the original specification at pages 2 and 3; neither appears to include this description. Claim 20 inherits this issue.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-4 and 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
In claim 1, it is unclear what “based on a plurality of crop characteristics” means. It appears the preceding elements are the characteristics, not something determined from the characteristics. See the 112-a rejection above.
In claim 7, The Markush group “a group consisting of dark with artificial light” seems to have a single element; to the extent this is the case it should be amended to read “…wherein the plurality of conditions includes dark with artificial light”.
Claim Rejections - 35 USC § 102 and/or 103
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 5-6, 8-10, 14-16 and 20 is/are rejected under 35 U.S.C. 102(a)(1) as anticipated by or, in the alternative, under 35 U.S.C. 103 as obvious over Koselka et al., U.S. patent publication 2006/0213167 (“Koselka”), cited in the non-final rejection of 10/1/20025.
With respect to Claim 5, Koselka shows a machine vision system mounted (Fig. 1, "Hand with stereo cameras"; para [0081]) to a mobile machine ("Scout platform") wherein the machine vision system includes an information capturing device (the stereo cameras); a computer having a processor and memory is connected to the information capturing device (Para [0104], "The platform may also house a computer, a communication device comprising a communications interface such as a cable connector or a wireless communication device and a GPS system. Two "Drive Wheels" may be utilized to propel the robot in one embodiment of the invention. "); the memory including stored crop and field information (Para. [0129], "The robot is configured to delineate the field. For example, coordinates for the corners of the field can be provided to the robot or visible landmarks such as posts or fences can be used for this purpose. The robot then logs either its position relative to a landmark or its GPS coordinates in the map at 403. In addition, the scout may gather information such as the size or ripeness of each piece of fruit at 405."; para. [0130], "Once a field is mapped that map is saved for future use, either in the same or successive growing seasons. The scout is configured to update the map for removed or added plants."). and positioning members mounted to and extending forward of the mobile machine. ("Arm Base", "Arm"; para [0107], the arm is pivotably mounted. "Forward" in this context lacks a reference point, either in the claim or in the robot shows in Fig. 1. As the arms extend beyond the perimeter of the "Scout Platform" and are rotatable and extendable any direction extending from the base is broadly "forward".
With further respect to Claim 5, [0131] shows adjustment of a harvesting plan based on the mapping of a field of fruit by the Scout. ("FIG. 5 illustrates an embodiment of a method of harvesting fruit with a harvester robot using a picking plan generated via a scout robot. First the scout maps the field at 501 as per FIG. 4. From the map, the scout creates a picking plan that includes the worker robot's path of travel through the field with details including the locations where the harvester is to stop around each plant at 502. The plan may include the order of fruit to pick with each arm and the approximate arm motions to reach each piece. Once the plan is complete, the scout transmits it to the appropriate worker at 503 (or to a server).") While the Scout of Fig. 1 is considered to inherently require "mounting a machine vision system" to a vehicle as the arms are already mounted, the embodiment of Fig. 10 is mounted to a tractor and described at paragraph [0117].
The resulting map is the processed information of the plant characteristics and crop and field information.
The resulting harvest (“the order of fruit to pick “) is also adjusted based on this processed information.
With respect to the limitations added by amendment, Koselka detects the color of the plants and/or fruits and the health of the plants. Fig. 4 #405; [0020] “Embodiments of the invention pre-map the individual fruit/vegetable size, color, and/or locations on the plant”; [0045] “While navigating through the grove and mapping, a scout robot can gather other useful information including the condition, size, quantity, health and ripeness of the fruit, individual trees and the orchards as a whole. In another embodiment, the scout robot can be equipped with a variety of sensors, including but not limited to cameras, hydration sensors, spectral sensors or filters to sense changes in coloration of the leaves, bark or fruit. “ Koselka also uses color of the crop as a threshold to determine ripeness ([0065], “The plan may include size or ripeness thresholds based on color or color pattern, such that only the ripe tomatoes are picked and the robot comes back the next day or week to pick the rest of the crop. Multi-spectral image analysis may be utilized by the system in order to determine whether a given piece of fruit is ripe or not, and the subtle differences in multi-spectral intensities of color may be preloaded into the robot for a given crop type.”). Koselka does not call the color detected by the digital camera a “color score”. However, digital camera reduce colors to a number, which is a score. Koselka compares the color to a threshold, which is a score. Applicant’s specification gives little additional detail; the scores for plant color, ear color and lodging are mentioned on pages 2 and 3 of the specification but little additional detail (i.e. how they are calculated, how they are used, etc.) is given. To the extent this simply means “color”, Koselka uses a digital camera which is understood to give a digital (i.e. numeric) color value. To the extent this means that thresholds are used to determine ripeness/health/growth stage or the like, Koselka gives an example (determining ripeness when harvesting tomatoes) of what the color data he collects are used for. It would have been obvious to one of ordinary skill in the art to use the color data collected in the other instances of Kolselka to evaluate the ripeness of the harvested crop and/or health of the plan (which is understood in an agricultural environment to be the productivity of the plant).
With respect to Claim 6, 8-10, 15-16 the relevant elements of the reference are identified above.
With respect to Claim 14, para. [0130] shows multiple inspections during a growing season. ("Once a field is mapped that map is saved for future use, either in the same or successive growing seasons. The scout is configured to update the map for removed or added plants.")
With respect to Claim 20, Koselka makes adjustments base on the captured information. (Para. [0129] "The scout then moves around the plant looking at the exterior with both the cameras…)
Claim Rejections - 35 USC § 103
Claims 1-4 and 11 are rejected under 35 U.S.C. 103 as being unpatentable over Koselka as applied to claim 5 above, and further in view of Wong et al., “Automated Corn Ear Height Prediction Using Video-Based Deep Learning”, hereinafter Wong. Koselka show the elements of Claim 1 as outlined above. Koselka shows robot vehicles with repositionable arms (e.g. 2 arms in Fig,. 1, 2 arms in Fig. 2). While it is unclear whether both of these arms extend forward from the vehicle in the position shown in those figures, the arms are rotatable. It would have been obvious to have both of the arms rotated to extend forward from the vehicle in order to inspect/harvest from a plant that is forward for the vehicle.
With respect to the newly added limitations, Koselka does not show determining a number of ears present on a plant, a height of an ear from the ground, a health of the ears, and an angle of a stalk in relation to a ground surface. Koselka shows a robot that is said to be usable for a variety of agricultural tasks. ([0012], “One approach for automated harvesting of fresh fruits and vegetables, pruning of vines, culling fruit, thinning of growth or fruit buds, selective spraying and or fertilizing, weeding, measuring and managing of agricultural resources is to use a robot comprising a machine-vision system containing cameras such as rugged solid-state digital cameras. The cameras may be utilized to identify and locate the fruit on each tree, points on a vine to prune, weeds around plants. In addition, the cameras may be utilized in measuring agricultural parameters or otherwise aid in managing agricultural resources.”) Koselka does not specify that the plant inspected is corn. Wong teaches that it is desirable to use a camera-equipped robot to take video of a corn plot and measure characteristics of the plants. (2373, “ Measurements can be taken using a camera fixed to the harvester or sprayer, or potentially via unmanned vehicles or drones.”). Wong teaches that desirable features to measure in corn include ear height (title and throughout), plant angle (lodging), and health (stay-green, intactness, ears on the ground, broken stalks) (2373, “The framework used here may be applied to measuring other important traits such as yield per plot, percentage of broken stalks or ears on the ground, root lodging, stalk lodging, stay-green and intactness.”) It would have been obvious to use the robot of Koselka to capture the video for the analysis of Wong in order to reduce the labor-intensive hand-measurement of corn crops. (Wong, 2371).
Wong does not mention counting ears per stalk.
McKay et al., US patent publication 2000400518 teaches that is desirable to use image recognition to count the number of ears on a corm plant. ([0120, “RPF mechanism could use the vision based camera system to scan the above ground portion of the plant to identify certain features using its image recognition capabilities such as counting number of ears on a corn plant for example”). It would have been obvious to count the number of ears of corn in the modified device of Koselka as taught by McKay et al. in order to identify desirable features of a corn plant.
With respect to Claim 3, Koselka teaches a camera (which is a sensor) as well as a stereo camera at [0077], and uses a wireless transmission system to interact with the computer network at [0104], which would be understood as a transceiver. ( from https://dictionary.cambridge.org/us/dictionary/english/transceiver, “a piece of equipment that can send out and receive electronic signals”)
Claim(s) 7 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Koselka as applied to Claim 5, above, in view of Kang et al., "Real-Time Fruit Recognition and Grasping Estimation for Robotic Apple Harvesting" ("Kang"). Koselka describes a device that uses cameras to identify fruit including (for example) apples at [0014], but does not elaborate on the conditions that the fruit is harvested under or what vision recognition algorithm is used. Kang et al. describes a computer vision system desirable for picking apples. See "3.2.2. Network Training": "There are 1200 images collected from different conditions to increase the diversity of the training data. For example, different time as day and night; different illumination as artificial lights, natural light, shadows, front lighting, side lighting and back lighting; different backgrounds as from the farms in Qingdao, China and Melbourne, Australia." Kang trains the vision algorithm on different lighting conditions including artificial light; therefore, it is logical that Kang contemplates viewing the items in the same range of conditions that the system uses to train. It would have been obvious to one of ordinary skill in the art to use the vision training/processing system of Kang in the device of Koselka (and to use it in the range of conditions for which the vision system was trained to work) in order to provide a "highly efficient and accurate" robotic harvesting system (Kang, abstract).
With respect to Claim 12, "optimal" is a term of degree. To the extent that Koselka as modified achieves a satisfactory scan of the crops it is considered broadly "optimal" as one image will necessarily be one where the ears of corn are most visible compared to the other images.
Claim(s) 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Koselka as applied to Claim 5, above, and further in view of Salisbury et al., U.S patent publication 2017/0273241. Claim 17 now requires at least one of flaps, agitators and an air knife. Applicant provides no illustration of a flap, and the two occurrences in the specification on pages 2 and 4 give no more detail. Salisbury shows detail of a vacuum end effector for an automated harvesting machine. From [0029], “FIGS. 34-35 show embodiments of a flexible flap usable in a vacuum harvesting system.” The use of the particular suction end effectors of Salisbury et al. in the device of Koselka would have been obvious to one of ordinary skill in the art in order to implement Koselka’s relatively general teaching (to use a suction harvesting device with a particular detailed suction harvester known in the art.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Koselka as applied to Claim 5, above. Koselka contemplates using one arm to move branches while the other performs some activity. In para [0016] Koselka moves a branch with a positioning member to allow further operations including picking fruit. In the same paragraph Koselka makes clear that different permutations of operations are desirable in different circumstances. (“An arm may be configured or coupled with an implement configured to pick, prune, cull, thin, spray, weed, take samples or perform any other agricultural task that is desired. Each arm may include one or more cameras and/or an embedded processor to accurately locate and reach each piece of fruit/vegetable, and an end effector which provides further action. The end effector may be a mechanical hand that grabs and picks fruit, or may contain some mechanical cutting or thinning device, some type of spraying mechanism, or any other device or implement to perform an agricultural function or observation or measurement. The end effector may also contain a mechanism to cut or snip the fruit from the stem rather then just pulling it free. The system may comprise two or more different style arms incorporated into the robot in order to reach the fruit on different parts of the tree or to perform different agricultural functions independent of the other arm or dependent upon the other arm, e.g., one arm may be configured to move branches so that another arm may be allowed to pick or cull fruit for example. “) Given Koselka’s description that a variety of operations are desirable, it would have been obvious to use one arm to move branches while the other arm inspects fruit to plan a harvest or perform a mid-season inspection. This would give meaning to Koselka’s “pick or cull fruit for example”.
Allowable Subject Matter
Claims 21 is allowed.
Claim 5 and dependents (with appropriate corrections to correct 112 problems) would be allowable if Claim 5 were amended to require all of the elements in the Markush grouping by deleting “at least one selected from the group consisting of” and if “plant productivity” were amended to “ear productivity”.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Kurtumus et al. and Karamim et al. describe computer vision algorithms that analyze ears of corn.
Anderson et al. describes a system that counts ears of corn in a field.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GREGORY A MORSE whose telephone number is (571)272-3838. The examiner can normally be reached M-F 7:30-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights