DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This a 2nd non-final rejection.
Response to Arguments
The amended claims in response to the rejection under 35 U.S.C. 101 have been considered but are unpersuasive. One of ordinary skill in the art could recognize objects being palm trees, the infections in a palm tree, and when it has been infected with Red Palm Weevil. An updated 101 rejection has been proposed as shown below.
Applicant’s arguments with respect to claim(s) 1, 5-18, 21-22 for the 35 U.S.C. 103 rejections have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Objections
Claim 13 recites “wherein a selecting of the ground-level images is based on spatial relationships between the multiple object locations and ground-level image acquisition locations”. “Wherein a” should be read as “wherein the”. Appropriate correction is required.
Claim 21 recites “that further stores instructions for training the classification machine learning process to search for deformations the palm tree crowns and ignore tree trunk image pixels”. “Deformations the palm tree crowns” should be read as “deformations in the palm tree crowns”. Appropriate correction is required.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1, 5-18, 21-22 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Claim 1 recites:
“performing an aerial images based (AIB) detection to find multiple object locations; wherein the performing of the AIB detection comprises applying an AIB detection machine learning process on aerial images” which can be reasonably interpreted as a human observer mentally detecting objects from an aerial image. The machine learning process is simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception.
“performing a ground-level based (GLB) detection of the objects, based on the multiple object locations; wherein the performing of the GLB detection comprises applying a GLB detection machine learning process on ground-level images” which can be reasonably interpreted as a human observer mentally detecting objects from an ground-level image. The machine learning process is simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception.
“and classifying, by a classification machine learning process, objects captured in the ground-level images to a plurality of classes, wherein the plurality of classes comprise the one or more given classes” which can be reasonably interpreted as a human observer mentally classifying objects they see into different classes. The classification machine learning process is simply appending well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception.
“and responding to the classifying, when finding one or more objects of the one or more given classes” which can be reasonably interpreted as a human observer using a physical aid such as pen and paper to write down the classes as a response to the classification.
“wherein the objects are palm trees, wherein the one or more given classes are one or more infected plant classes and wherein the one or more infected plant classes comprises a Red Palm Weevil infected class” which can be reasonably interpreted as a human observer mentally determining if objects are palm trees, if they are infected, and if they have a Red Palm Weevil infection.
Claim 5 recites “wherein the classification machine learning process was trained to search for deformations in palm tree crowns”. A person can mentally search for deformations/degradations in palm tree crowns, and a machine learning process is simply parts that are adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea.
Claim 6 recites “wherein the responding comprises predicting a future progress of Red Palm Weevil infection”. A person can mentally predict how bad the Red Palm Weevil infection can become in the future.
Claim 7 recites “wherein the responding comprises suggesting a ground-level image acquisition scheme based on the future progress of the Red Palm Weevil infection”. A person can mentally suggest how to capture images based on their predictions of the RPW infection.
Claim 8 recites “wherein the responding comprises suggesting a ground-level image acquisition scheme”. A person can mentally suggest how to capture ground-level images.
Claim 9 recites “wherein at least some of the aerial images belong to a vast aerial images database”. Images belonging to a database is a well-understood, routine, and conventional activity of a database.
Claim 10 recites “wherein at least some of the ground-level images belong to a vast ground-level images database”. Images belonging to a database is a well-understood, routine, and conventional activity of a database.
Claim 11 recites “wherein at least some of the ground-level images are street view images of a vast online street view images database”. Images belonging to a database is a well-understood, routine, and conventional activity of a database.
Claim 12 recites “wherein the ground-level images are selected out of sets of ground-level images”. A person can mentally select images out of a set of images.
Claim 13 recites “wherein a selecting of the ground-level images is based on spatial relationships between the multiple object locations and ground-level image acquisition locations”. A person can mentally select images based on the relationship between a location and an object’s location.
Claim 14 recites “wherein the sets of ground-level images are video streams”. A set of images are a well-understood, routine, and conventional activity of a video stream.
Claim 15 recites “wherein at least one of the AIB detection machine learning process, the GLB detection machine learning process, and the classification machine learning process is a deep learning process”. These machine learning processes being a deep learning process are well-understood, routine, and conventional activity of a deep learning process.
Claim 16 recites “wherein the objects are spread over one or more vast areas”. A person can mentally determine if objects are spread across a vast area.
Claim 21 recites “that further stores instructions for training the classification machine learning process to search for deformations the palm tree crowns and ignore tree trunk image pixels”. A person can mentally ignore tree trunks in an image and search for deformations only in the palm tree crown.
Claim 22 recites “wherein the performing of the (GLB) detection of the objects, based on the multiple object locations comprises determining the headings of one or more cameras that captured ground level based images of the objects”. A person can mentally determine the position of a camera for performing imaging on objects.
Claim 17 corresponds to claim 1, additionally reciting a detection system with processing circuits. These parts are adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea.
Claim 18 corresponds to claim 1. Thus, it is rejected for the same reasons as claim 1.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 9-13, 15-18, 22 are rejected under 35 U.S.C. 103 as being unpatentable over Perona (US 20170287170 A1) in view of Eldin.
Regarding claim 1, Perona discloses a non-transitory computer readable medium for detection of objects of one or more given classes (Perona, paragraph [0021], "System 100 includes a set of image data capture devices 110, a set of processing elements 120, and a set of output elements 130."), the non-transitory computer readable medium stores instructions for:
performing an aerial images based (AIB) detection to find multiple object locations (Perona, paragraph [0045], "FIG. 3 shows multi-view images (i.e., aerial images 305 and street view images 310), along with map data 315, which are analyzed to perform multi-view detection and recognition in order to generate a fine-grained geographic tree catalog 320"), wherein the performing of the AIB detection comprises applying an AIB detection machine learning process on aerial images (Perona, paragraph [0046], "The detection of the trees and the classification into different species is performed, in some embodiments, by detectors and classifiers (e.g., convolutional networks (CNNs), support vector machines (SVMs), etc.) that are trained on trees that are manually classified"),
performing a ground-level based (GLB) detection of the objects, based on the multiple object locations (Perona, paragraph [0045], "FIG. 3 shows multi-view images (i.e., aerial images 305 and street view images 310), along with map data 315, which are analyzed to perform multi-view detection and recognition in order to generate a fine-grained geographic tree catalog 320"),
wherein the performing of the GLB detection comprises applying a GLB detection machine learning process on ground-level images (Perona, paragraph [0046], "The detection of the trees and the classification into different species is performed, in some embodiments, by detectors and classifiers (e.g., convolutional networks (CNNs), support vector machines (SVMs), etc.) that are trained on trees that are manually classified"),
and classifying, by a classification machine learning process, objects captured in the ground-level images to a plurality of classes, wherein the plurality of classes comprise the one or more given classes (Perona, paragraph [0046], "The detection of the trees and the classification into different species is performed, in some embodiments, by detectors and classifiers (e.g., convolutional networks (CNNs), support vector machines (SVMs), etc.) that are trained on trees that are manually classified. In some embodiments, the classified trees of the training data are identified from existing tree inventory data from counties and cities that were acquired by professional arborists")
and responding to the classifying, when finding one or more objects of the one or more given classes (Perona, paragraph [0030], "Alternatively, or conjunctively, the fine-grained classification of some embodiments can be used to collect additional data about the geo-located elements, such as (but not limited to) a state (e.g., damaged, healthy, in need of maintenance, etc.), and/or size").
Perona does not teach “wherein the objects are palm trees, wherein the one or more given classes are one or more infected plant classes and wherein the one or more infected plant classes comprises a Red Palm Weevil infected class”.
However, Eldin teaches wherein the objects are palm trees, wherein the one or more given classes are one or more infected plant classes and wherein the one or more infected plant classes comprises a Red Palm Weevil infected class (Eldin, Page 2, "In [19], thermal images were used to test the ability to detect RPW in palm trees").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to replace one of Perona’s trees with a palm tree and detect RPW through thermal imaging, as taught by Eldin.
The suggestion/motivation for doing so would have been to identify and prevent degradation of a common pest in palm trees.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Perona in view of Eldin to obtain the invention as specified in claim 1.
Regarding claim 9, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1, wherein at least some of the aerial images belong to a vast aerial images database (Perona, paragraph [102], "Processes for detecting the geographic location of trees from multiple geo-referenced images and for performing fine-grained classification of the detected trees were performed using the Pasadena dataset. The dataset was obtained by downloading publicly available aerial and street view images from Google Maps™ at city-scale").
Regarding claim 10, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1, wherein at least some of the ground-level images belong to a vast ground-level images database (Perona, paragraph [102], "Processes for detecting the geographic location of trees from multiple geo-referenced images and for performing fine-grained classification of the detected trees were performed using the Pasadena dataset. The dataset was obtained by downloading publicly available aerial and street view images from Google Maps™ at city-scale").
Regarding claim 11, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1, wherein at least some of the ground-level images are street view images of a vast online street view images database (Perona, paragraph [0102], "Processes for detecting the geographic location of trees from multiple geo-referenced images and for performing fine-grained classification of the detected trees were performed using the Pasadena dataset. The dataset was obtained by downloading publicly available aerial and street view images from Google Maps™ at city-scale").
Regarding claim 12, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1, wherein the ground-level images are selected out of sets of ground-level images (Perona, paragraph [0102], "Processes for detecting the geographic location of trees from multiple geo-referenced images and for performing fine-grained classification of the detected trees were performed using the Pasadena dataset.").
Regarding claim 13, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 12, wherein a selecting of the ground-level images is based on spatial relationships between the multiple object locations and ground-level image acquisition locations (Perona, paragraph [0019], "Several embodiments of the invention provide vision-based systems that systematically detect, geo-locate and perform fine-grained classification for elements based on multi-view images (i.e., images captured from multiple different perspectives or viewpoints)").
Regarding claim 15, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1, wherein at least one of the AIB detection machine learning process, the GLB detection machine learning process, and the classification machine learning process is a deep learning process (Perona, paragraph [0028], "In many embodiments, the fine-grained classification is achieved by leveraging the power and flexibility of state-of-the-art detectors and classifiers (e.g., based on convolutional neural networks (CNNs))").
Regarding claim 16, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1, wherein the objects are spread over one or more vast areas (Perona, paragraph [0043], "FIG. 3 illustrates an example of analyzing multi-view images to captured image data for trees in an urban area").
Regarding claim 22, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1, wherein the performing of the (GLB) detection of the objects, based on the multiple object locations comprises determining the headings of one or more cameras that captured ground level based images of the objects (Perona, paragraph [0074], "In the illustrated embodiment, the process accepts geo-referenced images 605 from an aerial view, and multiple street view images captured from different viewpoints (as opposed to from the same viewpoint with different levels of zoom)").
Claim 17 corresponds to claim 1, additionally reciting a detection system, the system comprises one or more processing circuits that are configured to store instructions (Perona, paragraph [0028], “The system 100 also includes a set of processing elements 120 for analyzing image data (and other related data) to detect elements in the images, identify geographic locations for the detected elements, and perform a fine-grained classification of the geographically located elements”). Thus, they are rejected for the same reasons of obviousness as claim 1.
Claim 18 corresponds to claim 1. Thus, they are rejected for the same reasons of obviousness as claim 1.
Claim(s) 5 and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Perona (US 20170287170 A1) in view of Eldin and in further view of Hayrapetian (US 20190195478 A).
Regarding claim 5, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1.
Perona in view of Eldin does not teach “wherein the classification machine learning process was trained to search for deformations in palm tree crowns”.
However, Hayrapetian teaches wherein the classification machine learning process was trained to search for deformations in palm tree crowns (Hayrapetian, paragraph [0044], " For example, a captured image may be analyzed to determine if a plant is wilting, growing sideways, and/or has disease present on one or more leaves and/or branches).", tree crown comprises of leaves and branches, so this modification is checking to see if leaves are wilting/deforming).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to detect wilts in Perona’s (in view of Eldin) palm tree crown leaves, as taught by Hayrapetian.
The suggestion/motivation for doing so would have been to detect deformations and potential hazards in trees.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Perona in view of Eldin and in further view of Hayrapetian to obtain the invention as specified in claim 5.
Regarding claim 21, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1.
Perona in view of Eldin does not teach “that further stores instructions for training the classification machine learning process to search for deformations the palm tree crowns and ignore tree trunk image pixels”.
However, Hayrapetian teaches that further stores instructions for training the classification machine learning process to search for deformations the palm tree crowns and ignore tree trunk image pixels (Hayrapetian, paragraph [0044], " For example, a captured image may be analyzed to determine if a plant is wilting, growing sideways, and/or has disease present on one or more leaves and/or branches).", when a plant’s leaves are analyzed for wilting, tree trunks are not considered in the analysis).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to detect wilts in Perona’s (in view of Eldin) palm tree crown leaves, as taught by Hayrapetian.
The suggestion/motivation for doing so would have been to detect deformations and potential hazards in trees.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Perona in view of Eldin and in further view of Hayrapetian to obtain the invention as specified in claim 21.
Claim(s) 6-7 are rejected under 35 U.S.C. 103 as being unpatentable over Perona (US 20170287170 A1) in view of Eldin and in further view of Singh (US 20210279923 A1).
Regarding claim 6, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1.
Perona in view of Eldin does not teach “wherein the responding comprises predicting a future progress of Red Palm Weevil infection”.
However, Singh teaches wherein the responding comprises predicting a future progress of Red Palm Weevil infection (Singh, paragraph [0081], "Method 600 further includes applying 608 a machine learning algorithm to the trap data, the weather data, and the image data to generate predicted future pest pressure values at each of the plurality of pest traps").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to predict future pest pressures for RPW’s in Perona’s (in view of Eldin) images, as taught by Singh.
The suggestion/motivation for doing so would have been to determine where pests are located in the trees, leading to better precaution and awareness.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Perona in view of Eldin and in further view of Singh to obtain the invention as specified in claim 6.
Regarding claim 7, Perona in view of Eldin and Singh discloses the non-transitory computer readable medium according to claim 6.
Perona in view of Eldin and Singh does not teach “wherein the responding comprises suggesting a ground-level image acquisition scheme based on the future progress of the Red Palm Weevil infection”.
However, Singh additionally teaches wherein the responding comprises suggesting a ground-level image acquisition scheme based on the future progress of the Red Palm Weevil infection (Singh, paragraph [0082], "In addition, method 600 includes generating 610 a first heat map and generating 612 a second heat map").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to generate a heat map based on the pest pressure in Perona’s (in view of Eldin and Singh) image, as additionally taught by Singh.
The suggestion/motivation for doing so would have been to better visualize where the pests are located.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Perona in view of Eldin and Singh with the additional teachings of Singh to obtain the invention as specified in claim 7.
Claim(s) 8 are rejected under 35 U.S.C. 103 as being unpatentable over Perona (US 20170287170 A1) in view of Eldin and in further view of Kamon (US 20210076917 A1).
Regarding claim 8, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 1.
Perona in view of Eldin does not teach “wherein the responding comprises suggesting a ground-level image acquisition scheme”.
However, Kamon teaches wherein the responding comprises suggesting a ground-level image acquisition scheme (Kamon, paragraph [0106], “Display of a still image illustrated as examples in FIGS. 9A to 10 enables a user to check the still image, such as an image used for classification (discrimination), during diagnosis (observation), and to provide an instruction to capture an image again if the image has a fault, such as blur, halation, or fogging.”).
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to provide instructions to reshoot the image of Perona’s (in view of Eldin) if the quality is not sufficient, as taught by Kamon.
The suggestion/motivation for doing so would have been to accurately capture the trees in the image.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Perona in view of Eldin and in further view of Kamon to obtain the invention as specified in claim 8.
Claim(s) 14 are rejected under 35 U.S.C. 103 as being unpatentable over Perona (US 20170287170 A1) in view of Eldin and in further view of King (US 20200279374 A1).
Regarding claim 14, Perona in view of Eldin discloses the non-transitory computer readable medium according to claim 13.
Perona in view of Eldin does not teach “wherein the sets of ground-level images are video streams”.
However, King teaches wherein the sets of ground-level images are video streams (King, paragraph [0021], "The image capture devices 106(1)-106(N) may include a digital video camera or may be a still image camera configured to capture images periodically and/or on demand").
It would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to capture a video of Perona’s (in view of Eldin) trees, as taught by King.
The suggestion/motivation for doing so would have been to have a thorough view of the objects being captured.
Further, one skilled in the art could have combined the elements as described above by known methods with no change in their respective functions, and the combination would have yielded nothing more than predictable results.
Therefore, it would have been obvious to combine Perona in view of Eldin and in further view of King to obtain the invention as specified in claim 14.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WAYNE ZHANG whose telephone number is (571) 272-0245. The examiner can normally be reached Monday-Friday 10:00-6:00 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Sumati Lefkowitz can be reached on (571) 272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WAYNE ZHANG/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights