DETAILED ACTION
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine,
manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claim 1, and based upon consideration of all of the relevant factors with respect to the claim as a whole, claims 1-20 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101.
The Examiner will analyze Claim 1, and similar rationale applies to independent Claims 10 and 20.
The rationale, under MPEP § 2106, for this finding is explained below. The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a Process since the claim is directed to computer implemented method.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitation of “applying a machine learning model to the pre-processed image to determine a first estimate of a count of the plurality of objects” [mathematical concept]; “applying a density estimation algorithm to determine an object density of the region of interest” [mathematical concept/mental process]; and “adjusting the first estimate of the count of the plurality of objects based on the object density of the region of interest to obtain a final count of the plurality of objects” [mental process and/or mathematical concept] are directed to an abstract.
If/when the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The additional claim limitations “accessing an image of a plurality of objects” and “outputting the final count on a display.” is nothing more than insignificant extra solution activity.
A machine learning model are used to generally apply the abstract idea without limiting how it functions.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception. NO.
Furthermore, the generic computer components or machine learning algorithm of the processor/memory recited as performing generic computer or machine learning functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system.
The Examiner finds that Claims 2-9 does not state significantly more since the claim only recites additional steps for analyzing image using machine learning model in order to count objects.
Thus, claims 1-20 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.
Therefore, all claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 5, 10, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over GOTTUMUKKAL et al. (Pub. No. US 2024/0233130 hereinafter Got”) in view of Najibi et al. (Pub. No. US 2018/0285682).
Regarding claims 1, 10 and 20, Got teaches a computer implemented method comprising: Accessing (receiving) an image (first image) of a plurality of objects and pre-processing (formatting) the image (first image) to detect (identify) a region of interest in the image [Para. 9 “In example aspect includes a method for counting objects in an image, comprising receiving a first image having a first size greater than a size threshold.”; Para. 58 “At block 402, the method 400 includes receiving a first image having a first size greater than a size threshold”; Para. 59 “the method 400 includes formatting the first image into a second image having a second size less than the first size”; Para. 79 “In this optional aspect, at block 604, the method 400 may further include analyzing the density map to identify a cluster of objects”; Para. 82 “The computing device 200 may identify the region of interest 320 (FIGS. 3A and 3C) based on a comparison of object counts for the plurality of regions”; and Para. 84 “the computing device 200 may map the region of interest 320 of the second image 308 to the first image 304 to define the first region 305 in the first image 304 (FIG. 3A and 3B”];
the region of interest corresponding to a section of the image that includes the plurality of objects and applying a machine learning model (Object counting model) to the pre-processed image (the second image) to determine a first estimate (initial object count) of a count of the plurality of objects [Para. 61 “At block 406, the method 400 includes estimating, using a first object counting model, an initial object count in the second image”; Para. 62 “the estimating at block 406 may be generated using an ensemble of a YOLO-based object counting model and an object count estimation model.”; and Para. 67 “when the count of the detected objects in the second image 308 exceeds the object count threshold, the region of interest 320 in the first density map 312 (FIGS. 3A and 3C) is identified as including a cluster of objects and therefore having a higher object count as compared to other regions of the first density map 312. Then, the corresponding region 305 of the first image 304 (FIGS. 3A and 3B) is used to generate the third image 322 (FIGS. 3A and 3D)”];
applying a density estimation algorithm (density estimation) to determine an object density (density map) of the region of interest [Para. 70 “the determining component 245 may perform the task of density estimation for the third image 322. In an aspect, the determining component 245 may provide outputs, such as the second density map 324 (FIGS. 3A and 3E), that may be used to determine an updated count of detected objects 326 for the identified region of interest 320”];
adjusting (compiling) the first estimate (initial object count) of the count of the plurality of objects based on the object density (density map) of the region of interest to obtain a final count (updated object count) of the plurality of objects [Para. 70 “In an aspect, the determining component 245 may provide outputs, such as the second density map 324 (FIGS. 3A and 3E), that may be used to determine an updated count of detected objects 326 for the identified region of interest 320”]; and Para. 71 “the method 400 includes compiling an updated object count for the first image based on the updated first portion of the initial object count in the third image”];
outputting (transmitting) the final count (updated object count) [Para. 73 “the method 400 includes transmitting a notification based on the updated object count”; and Para. 76 “Examples of output devices may include, but are not limited to, a monitor or display screen, a speaker, a printer, and the like.”].
However, Got doesn’t explicitly teach displaying the final count as output.
Najibi teaches displaying the find count output. [Para. 29 “in some embodiments, the client device 110 may comprise a display module (not shown) to display information (e.g., in the form of user interfaces); Abstract “An indicator of the salient object count of the plurality of objects in the image is caused to be displayed on the user device” and Para. 184 “Operation 2230 is causing display, on the user device, of an indicator of the salient object count of the plurality of objects in the image”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got’s transmitting component 255 and outputting component 260 by incorporating Najibi’s teaching of ON A DISPLAY (display module) so that Got outputs the final count (updated object count) through user visible display interface instead of only transmitting a notification based on the updated object count. This medication improves Got by presenting the final count directly to the user, thereby improving real-time usability of the object counting result.
Regarding claims 5 and 15, Got teaches wherein the machine learning model (object counting model) is a trained deep learning model (baseline deep neural network) and the method includes using the trained deep learning model for determining the first estimate of count [Para. 61 and 63].
Got doesn’t explicitly teach the rest of claim limitations.
However, Najibi teaches wherein the machine learning model is a trained deep learning model trained on a public dataset of various objects and the method includes using the trained deep learning model [Para. 151, 142, 146] for: identifying candidate individual instances of an object from the plurality of objects [Para. 151, 142, 146]; applying a plurality of bounding boxes (bounding polygons such as boxes) to the image, a single bounding box being applied at each location where a candidate individual instance of the object is identified by the deep learning model [Para. 151, 142, 146]; and determining the first estimate of the count by counting the number of bounding boxes (number of ground truth bounding boxes) applied to the image [Para. 151, 142, 146].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got’s transmitting component 255 and outputting component 260 by incorporating Najibi’s teaching of ON A DISPLAY (display module) so that Got outputs the final count (updated object count) through user visible display interface instead of only transmitting a notification based on the updated object count. This medication improves Got by presenting the final count directly to the user, thereby improving real-time usability of the object counting result.
Claims 2 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over GOTTUMUKKAL et al. (Pub. No. US 2024/0233130 hereinafter Got”) in view of Najibi et al. (Pub. No. US 2018/0285682), and further in view of Himilton (Pub. No. US 20010041968).
Regarding claims 2 and 12, Got teaches wherein the image (first image) of the plurality of objects includes objects to be counted [Para. 58 and 67].
However, Got in view of Najibi doesn’t explicitly teach a background color that is complimentary to a color of the plurality of objects.
Himilton teaches objects includes a background color (changeable background color) that is complimentary (improved color contrast) to a color (color pills) of the plurality of objects (pills) [Para. 23].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s image capture environment by incorporating Hamilton’s teaching of background color (changeable background color) selected for complimentary contrast against color of the counted objects so that Got’s captured first image has improved object/background separation. This medication improves Got by increasing visual contrast for object detection and density estimation, thereby improving count accuracy.
Claims 3 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over GOTTUMUKKAL et al. (Pub. No. US 2024/0233130 hereinafter Got”) in view of Najibi et al. (Pub. No. US 2018/0285682) and further in view of Chen et al. (US Pub. No. US 2016/0253789).
Regarding claims 3 and 13, Got teaches wherein pre-processing the image comprises: reducing a size of the image [Para. 59 “At block 404, the method 400 includes formatting the first image into a second image having a second size less than the first size” and Para. 60 “For example, the formatting at block 404 may include performing image resizing process, so as to resize the high-resolution first image 304 into a low-resolution second image 308.”].
However, Got in view of Najibi doesn’t explicitly teach applying a noise removal algorithm to the image.
Chen teaches applying a noise removal algorithm (TNR method) to the image [para. 67].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s formatting component and pre-processing workflow by incorporating Chen’s noise removal algorithm before object count estimation on Got’s resized image. This medication improves Got by reducing image noise before object count and density map processing, thereby improving reliability of the detected object count.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over GOTTUMUKKAL et al. (Pub. No. US 2024/0233130 hereinafter Got”) in view of Najibi et al. (Pub. No. US 2018/0285682), and further in view of Wu et al. (US 2016/0206205).
Regarding claims 4 and 14, Got teaches wherein detecting a region of interest in the image further comprises: identifying a region containing the plurality of objects [Para. 80 and 82].
However, Got in view of Najibi doesn’t explicitly teach inspecting pixels of the image to determine which pixels of the image correspond to an object of the plurality of objects, and which pixels correspond to a background of the image.
However, Wu teaches inspecting pixels of the image to determine which pixels (foreground pixels) of the image correspond to an object of the plurality of objects, and which pixels (background pixels) correspond to a background of the image [Para. 85 and 83].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s ROI detection logic by incorporating Wu’s pixel level forground/background segmentation to classify Got’s ROI pixels as object associated or background associated. This medication improves Got by making the ROI boundary more precise before high resolution cropping, thereby reducing background interference in object counting.
Got also teaches determining a location of the image within which the plurality of objects is located [Para. 84].
However, Got in view of Najibi doesn’t explicitly teach determining, based on inspection of the pixels, a perimeter of a location of the image within which the plurality of objects are located.
Wu teaches determining, based on inspection of the pixels, a perimeter of a location of the image within which the plurality of objects is located [Para. 86 and 100].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s ROI detection logic by incorporating Wu’s pixel level forground/background segmentation to classify Got’s ROI pixels as object associated or background associated. This medication improves Got by making the ROI boundary more precise before high resolution cropping, thereby reducing background interference in object counting.
and
Got teaches a region of interest corresponding to the object containing location [Para. 54].
However, Got in view of Najibi doesn’t teach digitally drawing a border along the perimeter of the location, the border corresponding to the region of interest.
Wu teaches digitally drawing a border along the perimeter of the location, the border corresponding to the region of interest [Para. 88 and 114].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s ROI detection logic by incorporating Wu’s pixel level forground/background segmentation to classify Got’s ROI pixels as object associated or background associated. This medication improves Got by making the ROI boundary more precise before high resolution cropping, thereby reducing background interference in object counting.
Claims 6 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over GOTTUMUKKAL et al. (Pub. No. US 2024/0233130 hereinafter Got”) in view of Najibi et al. (Pub. No. US 2018/0285682), and further in view of Almbladh (Pub. No. US 2014/0185876).
Regarding claims 6 and 16. Got in view of Najibi doesn’t explicitly teach the claim limitations.
Almbladh teaches wherein the density estimation algorithm includes determining the object density of the region of interest by: identifying the plurality of objects in a foreground (motion region) of the image [Para. 12, and 15]; determining a total density (total area value) of the plurality of objects in the foreground of the image; identifying a single object of the plurality of objects [Para. 12, and 15]; determining a density of the single object (reference object area value) [Para. 12, and 15]; and dividing the total density (total are value) of the plurality of objects by the density of the single object (reference object area value) to obtain the object density of the region of interest [Para. 12, and 15].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s ROI density estimation workflow by incorporating Almbladh’s teaching of foreground area accumulation and division of total density by density of the single object so that Got computes the single object so that Got computes the dense region object quantity form total object region. This modification improves Got’s to count estimation where individual objects are difficult to resolve.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over GOTTUMUKKAL et al. (Pub. No. US 2024/0233130 hereinafter Got”) in view of Najibi et al. (Pub. No. US 2018/0285682), further in view of Almbladh (Pub. No. US 2014/0185876) and further in view of Moura (Pub. No. US 2020/0302781).
Regarding claims 7 and 17, Got teaches adjusting the first estimate (initial object count) of the count of the plurality of objects based on the object density (density map) of the region of interest (region of interest) to obtain a final count (updated object count) [Para. 71, and 70].
Got in view of Najibi further in view of Almbladh doesn’t explicitly teach the rest of claim limitations.
However, Moura teaches wherein adjusting the first estimate of the count of the plurality of objects comprises: modifying the plurality of bounding boxes such that the number of bounding boxes corresponds to a number of objects implied by the object density (density values) of the region of interest [Para. 59 and 72]; and recounting the plurality of bounding boxes to obtain the final count of the plurality of objects [Para. 59 and 72].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi further in view of Almbladh’s updated count compiling logic by incorporating Moura’s teaching of density-guided bounding boxes so that Got modifies box allocation or box sizing in dense ROI portions and then obtains the final count from the resulting number of bounding boxes. This medication improves Got’s final counting in dense or overlapping object regions.
Claims 8, 9, 11, 18 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over GOTTUMUKKAL et al. (Pub. No. US 2024/0233130 hereinafter Got”) in view of Najibi et al. (Pub. No. US 2018/0285682), further in view of Limer et al. (Pub. No. 20070189597).
Regarding claims 8 and 18, Got in view of Najibi doesn’t explicitly teach the claim limitations.
Limer teaches wherein the method further comprises: determining a measure of spread (touching or overlapping units) of the plurality of objects [Para. 65]; and assigning a scatter score to the image based on the measure of spread [Para. 65].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s ROI object-counting logic by incorporating Limer’s teaching of measuring spread using touching or overlapping unit and assigning a scatter score to the analyzed image. This modification improves Got’s reliability of density-based object counting.
Regarding claims 9 and 19. Got in view of Najibi doesn’t explicitly teach the claim limitations.
Limer teaches in response to the scatter score (likelihood of accurate counting) being below a threshold score (unacceptably low), automatically sending a control signal to a vibration device (vibrator 562) to vibrate a container (tray 504) holding the plurality of objects (Particles) to increase the spread of the plurality of objects (particles) [Para. 65, 66, 78, 130, and 131].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s ROI object-counting logic by incorporating Limer’s teaching of measuring spread using touching or overlapping unit and assigning a scatter score to the analyzed image. This modification improves Got’s reliability of density-based object counting.
Regarding claim 11, Got’s in view of Najibi doesn’t explicitly teach the claim limitations.
However, Limer teaches further comprising: a mobile workbench [fig. 4, 5 and related description]; a tray (try 504) on the mobile workbench for holding the plurality of objects (particles)), the tray including a vibration device (vibrator 562) that is in electronic communication with the processing circuit and configured to vibrate the tray to increase a scatter (un-stacked) of the plurality of objects [Para. 130, and 131]; a light source to illuminate the plurality of objects (units) [Para. 11]; and an imaging device (image acquisition component) in communication with the processing circuit, wherein the imaging device is configured to capture the image of the plurality of objects (units) and send the image to the processing circuit [Para. 10]; wherein the tray, and therefore the image of the plurality of objects, includes a matte background color that is complimentary to a color of the plurality of objects [fig. 4, 5 and related description].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Got in view of Najibi’s ROI object-counting logic by incorporating Limer’s teaching of measuring spread using touching or overlapping unit and assigning a scatter score to the analyzed image. This modification improves Got’s reliability of density-based object counting.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-7 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, O’Neal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-0101 (IN USA OR CANADA) or 571-272-1000.
/SOLOMON G BEZUAYEHU/ Primary Examiner, Art Unit 2666
Prosecution Insights