Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
This action is in response to the applicant's communication filed on 11/25/2024. In virtue of this communication, claims 1-20 filed on 11/25/2024 are currently pending in the instant application.
Information Disclosure Statement
The information Disclosure statement (IDS) form PTO-1449, filed on 03/11/2025 are in compliance with the provisions of CFR 1.97. Accordingly, the information disclosed therein was considered by the examiner.
Drawings
The drawings received on 11/25/2024 have been reviewed by Examiner and they are acceptable.
Double Patenting
The non-statutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper time wise extension of the "right to exclude" granted by a patent and to prevent possible harassment by multiple assignees. See In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed.Cir.1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); and In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on a nonstatutory double patenting ground provided the reference application or patent either is shown to be commonly owned with this application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO internet Web site contains terminal disclaimer forms which may be used. Please visit http://www.uspto.gov/forms/. The filing date of the application will determine what form should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about e-Terminal Disclaimers, refer to http://www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Instant independent claims 1, 14, 19 are rejected under the judicially created doctrine of obviousness-type double patenting as being unpatentable over corresponding similar independent claims 1(method), 14(system), and 18 (computer readable medium) of US Patent No. 11,803,959, similarly, independent claims 1(method), 13(system), and 19 (computer readable medium) of US Patent No. 12/190,501 The conflicting claims are not identical because the embodiments of co-owned claims omit steps not explicitly required by the embodiment of instant claims. However, the conflicting claims are not patentably distinct from each other because:
· Instant claims and co-owned claims recite common subject matter;
· Instant claims which recite the open ended transitional phrase “comprising,” does not preclude the difference in steps recited by co-owned claims, and
· the elements of instant claims are obvious over co-owned claims, and completely anticipate the subject matter of instant claim, and “anticipation is the epitome of obviousness” Connell v. Sears, Roebuck & Co.,722 F.2d 1542, 1548, 220 USPQ 193, 198 (Fed. Cir. 1983) (citing In re Fracalossi, 681F.2d 792, 215 USPQ 569 (CCPA 1982)).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 4, 6, 14, 17, and 19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shulman (US 2018/0373937), further in view of Redden et al. (US 2018/0330166).
As per claim 1, A client device comprising: “a network interface subsystem to obtain a first image that captures a depiction of an individual plant within a plurality of plants; instructions that, in response to execution by one or more processors, cause the one or more processors to execute a model using the first image as an input,”(Shulman,¶[0027] ¶[0029], ¶[0059] discloses selection of individual plant within a plurality of plants. Device receives input from an operator to select a certain plant, or to select certain parameters associated with a plant, or to perform comparisons between the most recent plant data and archived plant data.)
“and a display to present a change to the individual plant over time, wherein the instructions cause the one or more processors to determine visuals to present on the display based on the identifier of the individual plant and one or more previously-captured images that include the individual plant.”(Shulman, ¶[0047] discloses displaying a current image of a plant or displaying an archived image of a plant which corresponds to plant identifier. take a picture of a plant corresponding to plant identifier, and compare the image to the image previously captured images.¶[0048] discloses archiving the same plant.)
However Shulman does not explicitly disclose the following which would have been obvious in view of Redden from similar filed of endeavor “instructions that, in response to execution by one or more processors, cause the one or more processors to execute a trained machine learning model using the first image as an input,” “wherein the execution of the trained machine learning model generates an identifier of the individual plant;” (Redden, ¶[0008] discloses individual plant localization, ¶[0055] discloses identify bounding boxes that specify where plants/crops/weeds/species are physically located on the ground in the field as represented by the image data and the types of the plants within each bounding box. per-image box descriptors. ¶[0059] discloses the model is trained by inputting the labeled training data, including both the image data and the labels into the function (or set of functions representing the model). The parameter values are then learned, and are stored in conjunction with the function/s. The model is stored in the computer's 140 memory, and may be accessed when used, for example when the platform 100 is driving through the field. In use, new image data is received and is input into the model, that is to say into the function and associated parameter values, and an output is generated that represents the model's. learned parameter inference on an input image.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Redden technique of automated plant detection using image data into Shulman technique to provide the known and expected uses and benefits of Redden technique over automated farming technique of Shulman. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Redden into Shulman in order to achieve precision application of plant treatment which is not expensive and time consuming. (Refer to Redden ¶[0002-0003].)
Claims 14 and 19 have been analyzed and are rejected for the reasons indicated in claim 1 above.
As per claim 4, The client device of claim 1, “wherein the instructions cause the one or more processors to present a user interface on the display that shows disease progression associated with the individual plant.”(Shulman, ¶[0060] discloses displaying an image of an individual plant with a plant identifier, device can display the previous harvest for the plant, growth of the plant the previous year to date in the growing cycle, comparison of current hydration levels to archived hydration levels, comparison of current mold level parameters to archived mold level parameters, a comparison of anthocyannis, flavonoids, acidity, brix/sugar, chlorophyll, carotenoids, senescence, water stress, nitrogen deficiency, gaseous pollutants, fungal infections viral infections to previous archived data for the plant.)
Claim 17 has been analyzed and is rejected for the reasons indicated in claim 4 above.
As per claim 6 The client device of claim 1, “wherein the client device is one of a mobile phone, tablet, virtual reality apparatus, an augmented reality apparatus, or an in-vehicle navigation system.” (Shulman, ¶[0029] discloses mobile collection unit 108 may be a mobile phone, tablet, laptop, or other mobile processing device with a camera and processing capability.)
Claim(s) 5 and 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shulman (US 2018/0373937), further in view of Redden et al. (US 2018/0330166), further in view of Philibin et al. (US 2016/0180151).
As per claim 5, The client device of claim 1, Shulman as modified by Redden further discloses “wherein: the identifier is a first identifier of the individual plant that corresponds to the first image;” (Shulman, ¶[0048] discloses the images associated with one plant identifier or location.) “and before causing the display to present the change to the individual plant over time,” (Shulman, ¶[0041] discloses Images of the same plant taken during subsequent passes through the vineyard can show changes over time such as growth rates. ¶[0047-0048] discloses displaying a current image of a plant or displaying an archived image of a plant, take a picture of a plant corresponding to plant identifier 304, and compare the image to the image downloaded from computer. ¶[0060] discloses displaying growth of the plant the previous year to date in the growing cycle .)
However Shulman as modified by Redden does not explicitly disclose the following which would have been obvious in view of Philibin form similar field of endeavor “the instructions cause the one or more processors to: execute the trained machine learning model using the previously-captured images as inputs, wherein the execution of the trained machine learning model generates a plurality of identifiers of the individual plant that correspond to the previously-captured images; and associate the first identifier with the plurality of identifiers.”(Philibin, ¶[0016] discloses processing each of a plurality of images using the neural network in accordance with the trained values of the parameters of the neural network to determine a respective numeric embedding of each of the plurality of images; receiving a new image; processing the new image using the neural network in accordance with the trained values of the parameters of the neural network to determine a numeric embedding of the new image; and classifying the new images as being an image of the same object as one or more of the plurality of images from distances between the numeric embedding of the new image and numeric embeddings of images from the plurality of images. ¶[0031] discloses the numeric embedding system 120 processes each image in a set of multiple images of objects of the particular object type using the neural network 120 to determine a respective numeric embedding of each of the images. The numeric embedding system 120 then receives a new image and processes the new image using the neural network 120 to determine a numeric embedding of the new image. The numeric embedding system 120 then determines whether the new image is an image of the same object as any of the images in the set of multiple images by comparing the numeric embedding of the new image with the numeric embeddings of the images in the set of images.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Philibin technique of using neural network for image object detection into Shulman as modified by Redden technique to provide the known and expected uses and benefits of Philibin technique over automated farming technique of Shulman as modified by Redden. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement. Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Philibin into Shulman as modified by Redden in order to accurately extract features and identify objects in input images . (Refer to Philibin ¶[0003-0004].)
Claim 18 has been analyzed and is rejected for the reasons indicated in claim 5 above.
Claim(s) 2, 3, 13, 15, 16 , and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shulman (US 2018/0373937), in view of Redden et al. (US 2018/0330166), further in view of Itzhaky et al. (US 2016/0148104).
As per claim 2, The client device of claim 1, However Shulman as modified by Redden does not explicitly disclose the following which would have been obvious in view of itzhaky form similar field of endeavor “wherein the instructions cause the one or more processors to present a user interface on the display that includes a time-lapsed sequence of the individual plant.”(Itzhaky, ¶[0026] discloses The test image sequence may be a series of images captured sequentially from the same viewpoint with substantially similar optical characteristics. The images in the test image sequence may be captured periodically. Further, the time intervals between captured images may be sufficient to demonstrate stages of plant development.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Itzhaky technique of plant monitoring into Shulman as modified by Redden technique to provide the known and expected uses and benefits of Itzhaky technique over automated farming technique of Shulman as modified by Redden. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Itzhaky into Shulman as modified by Redden in order to accurately identify and monitor the plants in timely manner . (Refer to Itzhaky ¶[0008].)
Claim 15 has been analyzed and is rejected for the reasons indicated in claim 2 above.
As per claim 3, The client device of claim 2, “wherein the instructions cause the one or more processors to: predict a growth rate or yield of the individual plant based on the time-lapsed sequence;”(Itzhaky, ¶[0065] discloses The labeled training set 510 also includes training outputs such as a time to harvest label 513 indicating a yield of the plant at a future harvest time. The yield of the plant may be expressed as a plant yield at the harvest time. For example, the plant yield may be measured based on a quantity of plant parts (e.g., fruits), a total weight of yield, a total volume of yield, and so on.¶[0068] discloses prediction from test image sequences. )
“ and cause the display to present the prediction.”(Shulman, ¶[0060] discloses yield information display.¶[0064] discloses future yield management.)
Claim 16 has been analyzed and is rejected for the reasons indicated in claim 3 above.
As per claim 13, The client device of claim 1, However Shulman as modified by Redden does not explicitly disclose the following which would have been obvious in view of itzhaky form similar field of endeavor “wherein the instructions cause the one or more processors to execute the trained machine learning model using both the first image and a time interval since a milestone in a life of the individual plant as inputs.” (Itzhaky, ¶[0077] discloses the input structure for an image input may include an image parameter and an image frequency parameter. The image parameter may indicate an amount of successive images in an image sequence. The image frequency parameter may indicate one or more time intervals between successive captures of images of an input. The time intervals may be the same (e.g., when images are captured periodically) or different. ¶[0080] discloses Each test input may have the same input structure as one or more training inputs. Thus, the test input may have the same number of input parameters as that of a training input, and the parameters may be taken at similar time intervals.)
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shulman (US 2018/0373937), in view of Redden et al. (US 2018/0330166), further in view of Perona et al. (US 2017/0287170.)
As per claim 7, The client device of claim 1, Although Shulman as modified by Redden discloses “generating bounding boxes for individual plant” (Redden, ¶[0049]¶[0055]), however it does not explicitly disclose the following which would have been obvious in view of Perona from similar filed of endeavor “wherein the instructions cause the one or more processors to: process the first image to generate a bounding shape that corresponds to the individual plant; and execute the trained machine learning model using both the first image and the bounding shape as inputs.” (Perona, ¶[0050] discloses where each bj=(xj,yj,wj,hj) is a bounding box and sj=CNN (X,bj;γ) is a corresponding detection score over CNN features extracted from image X at location bj. The ROI proposals can be understood as a short list of bounding boxes that might contain valid detections. ¶[0053] discloses generating region proposal. )
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Perona technique of trees locating and monitoring into Shulman as modified by Redden technique to provide the known and expected uses and benefits of Perona technique over automated farming technique of Shulman as modified by Redden. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement. Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Perona into Shulman as modified by Redden in order to reduce cost and time of locating and monitoring trees for planning. (Refer to Perona ¶[0003-0004].)
Claim(s) 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shulman (US 2018/0373937), in view of Redden et al. (US 2018/0330166), in view of Perona et al. (US 2017/0287170), further in view of Liang et al. "Conventional and hyperspectral time-series imaging of maize lines widely used in field trials." Gigascience 7.2 (2018).
As per claim 8, The client device of claim 7, However Shulman as modified by Redden as modified by Perona does not explicitly disclose the following which would have been obvious in view of Liang form similar field of endeavor “wherein the bounding shape is the smallest size possible that encloses outer extremities of the individual plant.” (Liang, Figure 1, caption discloses Minimum bounding rectangle of plant pixels is shown in red)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Liang technique of field monitoring into Shulman as modified by Redden as modified by Perona technique to provide the known and expected uses and benefits of Liang technique over automated farming technique of Shulman as modified by Redden as modified by Perona. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Liang into Shulman as modified by Redden as modified by Perona in order to provide accurate monitoring of filed to increase production. (Refer to Liang page 2, Col. 1, first line.)
As per claim 9, The client device of claim 7, Shulman as modified by Redden as modified by Perona as modified by Liang further discloses “wherein the bounding shape encloses a predetermined percentage of the individual plant.” (Liang, Figure 1, section A, discloses 100% coverage of plant. It is predetermined by all the pixel enclosure rather that calculated after observing an arbitrary box.)
Claim(s)10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shulman (US 2018/0373937), in view of Redden et al. (US 2018/0330166), in view of Perona et al. (US 2017/0287170), further in view of Tagez et al. (US 10,825168).
As per claim 10, The client device of claim 7, however Shulman as modified by Redden does not explicitly disclose the following which would have been obvious in view of Tagez form similar filed of endeavor “wherein: the trained machine learning model is a first trained machine learning model; and the instructions cause the one or more processors to generate the bounding shape by executing a second trained machine learning model.” (Tagzes, Col. 15, line 12-20 discloses the bounding box generator determines organ/region boundaries in axial, coronal, and sagittal directions. Certain examples detect the boundaries and form a bounding box based on slice-level classification using a trained deep convolutional network (e.g., trained using a Caffe deep learning framework, etc.). line 31-35, the bounding box generator generates one or more bounding boxes around item(s) of interest in the image based on slice classifier output from the network. Col 31, line 21, discloses using a first machine learning model to identify a region of interest in the anatomy or including the anatomy. Lines 24-27 discloses a bounding box generator to generate a bounding box around the region of interest using a second machine learning model and to provide image data within the bounding box.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Tagze technique of using multi machine learning models into Shulman as modified by Redden as modified by Perona technique to provide the known and expected uses and benefits of Tagze technique over automated farming technique of Shulman as modified by Redden as modified by Perona. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement. Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Tagze into Shulman as modified by Redden as modified by Perona in order to accurately detect objects in an image. (Refer to Tagze col.1 line 49.)
Claim(s)11 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shulman (US 2018/0373937), in view of Redden et al. (US 2018/0330166), in view of Araújo et al. "Fine-grained hierarchical classification of plant leaf images using fusion of deep models." 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI). IEEE, 2018.
As per claim 11, The client device of claim 1, however Shulman as modified by Redden does not explicitly disclose the following which would have been obvious in view of Araújo form similar filed of endeavor “wherein: the trained machine learning model is a first trained machine learning model; and the instructions cause the one or more processors to: execute the first trained machine learning model in response to a determination that the plurality of plants are a first genus or species of plant; and execute a second trained machine learning model in response to a determination that the plurality of plants are a different genus or species of plant, wherein the execution of both the first trained machine learning model and the second trained machine learning model generate an identifier of the individual plant.”( Araújo, page 3, Col.1, section C discloses the genus predicted by the first level (coarse class) (3-C) is used to select the correct CNNs trained to consider the corresponding species. Figure 3, shows heretical arrangement.)
Before the effective filing date of the claimed invention it would have been obvious to a person of ordinary skill in the art to combine Araújo technique of using hierarchical machine learning models for plant detection in images into Shulman as modified by Redden technique to provide the known and expected uses and benefits of Araújo technique over automated farming technique of Shulman as modified by Redden. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement. Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Araújo into Shulman as modified by Redden in order to accurately detect plant classes and monitor the plant . (Refer to Araújo page 1, Col 2.)
Claim(s)12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Shulman (US 2018/0373937), in view of Redden et al. (US 2018/0330166), in view of Tang et al. (US 2016/0189010).
As per claim 12, The client device of claim 1, however Shulman as modified by Redden does not explicitly disclose the following which would have been obvious in view of Tang form similar filed of endeavor “wherein the instructions cause the one or more processors to execute the trained machine learning model using both the first image and a position coordinate indicative of a location of the individual plant as inputs.” (Tang, ¶[0041] discloses The particular image and the location information associated with the particular image can be inputted, by the location-based object recognition module , into the neural network. Further see ¶[0051], then, ¶[0061] discloses The image and its associated location information can be inputted into the neural network . )
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine Tang technique of image object recognition into Shulman as modified by Redden technique to provide the known and expected uses and benefits of Tang technique over automated farming technique of Shulman as modified by Redden. The proposed combination would have constituted a mere arrangement of old elements with each performing their known function, the combination yielding no more than one would expect from such an arrangement.
Therefore, it would have been obvious to a person of ordinary skill in the art to incorporate Tang into Shulman as modified by Redden in order to accurately detect and recognize objects in images . (Refer to Tang ¶[0003].)
Contact
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAGHAYEGH AZIMA whose telephone number is (571)272-1459. The examiner can normally be reached Monday-Friday, 9:30-6:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at (571)272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAGHAYEGH AZIMA/Examiner, Art Unit 2671
Prosecution Insights