DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election of Species I (claims 3 and 7) in the reply filed on 08/05/2026 is acknowledged. Applicant’s election is noted with traverse.
Although Applicant argues that under MPEP § 803 and MPEP § 808.01(a) the Office must demonstrate patentable distinction between the species, the present application is an international stage application under 35 U.S.C. 371 and unity of invention is governed by 37 CFR 1.475 and PCT Rule 13. The species are set forth to independent and distinct technical features such that the search is diverse for each species ( Species I, depicted by paragraph [0017] and defined by claims 3 and 7, is directed to detecting whether each animal is standing or lying down using an object detection model with neural networks trained to classify standing and lying animals directly from multi-camera image data; whereas Species II, depicted by paragraph [0018] and defined by claims 4–5 and 8–9, is directed to posture detection via a geometric comparison of bounding box position and/or orientation against predefined animal lying subareas such as cubicles ).
Thus, while the restricted species may be disclosed as usable together in the same overarching multi-camera animal husbandry tracking system, the species are directed to different underlying concepts and different technical approaches: a trained machine-learning classification methodology using a neural-network object detection model, and a geometric, rule-based methodology using spatial comparison of bounding box position and/or orientation against a predefined animal lying subarea. A search directed to one species would not necessarily be expected to identify the most relevant prior art for the other species. For example, a search for trained neural network object detection architectures and posture classification models (CPC classes G06V 10/82, G06V 40/103) would not necessarily locate the most relevant prior art for geometric bounding box position and orientation comparisons relative to stall or cubicle layout boundaries; likewise, a search for barn layout coordinate-mapping and animal lying subarea geometric thresholding (CPC classes A01K 11/006, A01K 29/005, G06T 7/73) would not necessarily locate the most relevant prior art for neural network training classifiers for standing and lying animals.
Accordingly, the search for the generic claims would not reasonably encompass the specific subject matter of each dependent subcombination or species, and examination of all species together would impose a serious search and examination burden. For at least these reasons, and upon reconsideration of Applicant’s traversal, the requirement for unity of invention is still deemed proper and is therefore made FINAL.
Accordingly, examination will proceed on the elected invention and generic claims only (claims 1–3, 6, and 7). The application has pending claims 1–9 (non-elected claims 4, 5, 8, and 9 are withdrawn from further consideration pursuant to 37 CFR 1.142(b)).
Specification
The abstract of the disclosure is objected to because it is not presented entirely in clear narrative form and it employs legal phraseology often used in claims, specifically the terms "means" (e.g., "monitoring and analyzing means," "image processing means"). MPEP § 608.01(b) advises that the abstract should avoid legal phraseology and indefinite words or phrases such as "means" and "said" that are often found in patent claims, and should instead employ concise, technically descriptive language. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
Such claim limitations are: “monitoring and analyzing means configured for repeatedly generating images and gathering data” in claim 1, and an “image processing means” in claims 1–3, 6, and 7. These limitations use the term “means,” are modified by functional language, and do not recite sufficiently definite structure for performing the complete recited functions. Accordingly, these limitations are interpreted under 35 U.S.C. 112(f) as covering the corresponding structure disclosed in the specification for performing the respective claimed functions and equivalents thereof. Claims 2, 3, 6, and 7 inherit this interpretation through their dependency from claim 1.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112(b)
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1–3, 6, and 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim limitation “image processing means” invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. Where a claim limitation invokes 112(f), the specification must disclose the specific structure, material, or acts, including an algorithm sufficient to perform the entire claimed function, for each function recited. See MPEP § 2181(II)(B); Aristocrat Techs. Austl. Pty Ltd. v. Int'l Game Tech., 521 F.3d 1328 (Fed. Cir. 2008); Blackboard, Inc. v. Desire2Learn, Inc., 574 F.3d 1371 (Fed. Cir. 2009). The specification fails to disclose sufficient corresponding structure, including any algorithm, for at least the recited functions of "detecting at least one silhouette of an animal" and "defining an animal bounding box around each animal silhouette." Paragraph [0038] states only that "at that level, a silhouette of each animal 2 in each image is detected and an animal bounding box is defined around each animal silhouette in each image by means of the image processing means," and further states "this will be explained in further detail below." However, no further detail describing how the silhouette detection or bounding box definition is performed is thereafter provided in the specification. The specification instead proceeds directly to a discussion of height-dependent projection (paragraphs [0039–0042]) without ever returning to explain the promised algorithm for silhouette detection or bounding box definition. Because the specification discloses no algorithm, technique, or other sufficiently definite structure for performing these two recited functions, one of ordinary skill in the art cannot ascertain the scope of the "image processing means" limitation with reasonable certainty, rendering claims 1–3, 6, and 7 indefinite. Therefore, the claims are indefinite and are rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claims 1–3, 6, and 7 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being incomplete for omitting essential elements, such omission amounting to a gap between the elements. See MPEP § 2172.01. The omitted elements are: The claim does not define what threshold, metric, or criteria of "comparison" constitutes an identification of an "identical animal" for the step of “identifying identical animals in the stitched image by comparing the projected animal bounding boxes in the image stitching plane” in claim 1. For example: degree of bounding box overlap, center-point Euclidean proximity, or Intersection-over-Union threshold.etc. Because claims 2–3, 6, and 7 depend from claim 1, they inherit this ambiguity, fail to cure the deficiency.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1–3, 6, and 7 are rejected under 35 U.S.C. § 101 because the claimed invention is directed to a judicial exception without significantly more.
This rejection has been made in accordance with the current USPTO subject matter eligibility framework, including MPEP §§ 2103–2106.07, the 2019 Revised Patent Subject Matter Eligibility Guidance, the October 2019 Patent Eligibility Guidance Update, the 2024 Guidance Update on Patent Subject Matter Eligibility, Including on Artificial Intelligence, the July 2024 AI Subject Matter Eligibility Examples, the August 4, 2025 USPTO memorandum titled "Reminders on evaluating subject matter eligibility of claims under 35 U.S.C. § 101," and the USPTO's guidance concerning Ex parte Desjardins, Appeal No. 2024-000567. The claims have been evaluated under the broadest reasonable interpretation, and the claims have been considered as a whole.
Step 1: Statutory category
Independent claim 1 is directed to an animal husbandry system comprising monitoring/analyzing means, cameras, and image processing means, and therefore falls within the statutory category of a machine or manufacture. Accordingly, the analysis proceeds to Step 2A.
Step 2A, Prong One (Judicial exception)
Independent claim 1 recites: detecting at least one silhouette of an animal in a first image and in a second image; defining an animal bounding box around each animal silhouette; detecting whether each animal is standing or lying down; applying to each animal bounding box either a first projection onto an image stitching plane in case the animal is standing, or a second projection onto the image stitching plane in case the animal is lying down; stitching the projected animal bounding boxes in the image stitching plane; and identifying identical animals in the stitched image by comparing the projected animal bounding boxes in the image stitching plane.
These limitations recite an abstract idea, namely mathematical concepts (geometric coordinate transformations, parallax projections, and spatial comparison algorithms), mental processes (evaluating whether an animal is standing or lying down, and visually matching animals across viewpoints), and certain methods of organizing human activity (animal inventory and tracking).
The claim recites information collection, data analysis, mathematical coordinate transformations, and spatial comparisons. The "images", "silhouettes", "bounding boxes", "first and second projections", "image stitching plane", and "stitched image" are used as mathematical constructs and data representations in an image-analysis pipeline. The claim does NOT recite an improvement to the physical camera sensors, lens assemblies, or optical image-sensing hardware itself. Rather, the claim uses generic cameras and computer processing means to capture conventional image data and execute mathematical coordinate translations and geometric comparisons.
The claim is similar in character to claims that courts have found abstract where the focus is collecting data, analyzing the data mathematically, and determining an outcome based on the analysis. In Electric Power Group, LLC v. Alstom S.A., the Federal Circuit recognized that collecting, analyzing, and displaying information are abstract ideas. The present claims similarly collect multi-camera image data, analyze pixel boundaries to extract bounding boxes, classify animal posture, mathematically project bounding box coordinates onto a 2D plane based on the classification, and compare coordinate overlap to determine object identity. Furthermore, observing an animal, recognizing whether it is standing or lying down, and visually correlating its identity across two viewpoints are fundamental mental processes historically performed by human farm workers.
Claim 2 further limits the image stitching plane to "a floor level of the area", which merely defines an abstract geometric reference plane in a coordinate space.
Claims 3 and 7 further recite detecting posture by "using an object detection model with neural networks, trained to classify standing and lying animals from images". These limitations recite the use of a mathematical model and statistical machine-learning algorithm as a tool for performing the abstract classification task. The claims do not recite any specific, unconventional neural network architecture or technical improvement to machine-learning hardware or computational architecture itself.
Claim 6 further recites identifying identical animals by "comparing the projected animal bounding boxes in the image stitching plane using a bipartite matching algorithm". This limitation recites an explicit mathematical graph-matching formula (e.g., Hungarian/Munkres algorithm) to calculate optimal pairing between coordinate sets.
The claims are also consistent with the reasoning of AI Visualize, Inc. v. Nuance Communications, Inc., where the Federal Circuit affirmed ineligibility where claims were directed to manipulating and projecting image data at a high level of generality rather than to a specific technical improvement in computer or imaging functionality. Here, the character of the elected claims as a whole is video data collection, silhouette/bounding box extraction, geometric plane projection, and mathematical graph matching, not an improvement to camera hardware, computer memory, or network infrastructure.
Accordingly, claims 1–3, 6, and 7 recite an abstract idea under Step 2A, Prong One.
Step 2A, Prong Two (Practical Application)
The additional elements, considered individually and in combination, do not integrate the abstract idea into a practical application.
The recited "plurality of cameras", "monitoring and analyzing means", and "image processing means"amount to generic data-gathering hardware and conventional computing processors performing routine computational tasks.
The claims do not recite a particular improvement to computer, sensor, or imaging technology. They do not improve the physical optics, shutter mechanisms, or physical sensor arrays of the cameras. The cameras merely capture standard overhead video frames that serve as input data for mathematical plane projections.
The claims further do not recite an improvement to artificial-intelligence or machine-learning technology itself. The claims do not train a neural network model in a novel way that improves processing efficiency, modify neural network architecture to reduce parameter storage, or improve inference execution time through a specific claimed hardware/software mechanism. The "object detection model with neural networks" is recited purely functionally as an off-the-shelf mathematical classifier for standing versus lying animals.
This analysis is consistent with the USPTO's 2024 AI subject matter eligibility guidance and examples, which emphasize that AI claims are eligible when they recite a specific technological improvement to AI or integrate the exception into a practical application, rather than merely invoking generic AI/neural networks to perform data classification.
This case is distinguishable from Ex parte Desjardins. In Desjardins, the claims recited a specific structural machine-learning training architecture that preserved prior task knowledge and reduced computational complexity. Here, the claims do NOT recite any specific parameter update, attention mechanism, or resource-saving structure that improves the computational operation of the neural network or processor. Nor does limiting the abstract idea to the field of animal husbandry or barn monitoring render the claims eligible. In Recentive Analytics, Inc. v. Fox Corp., the Federal Circuit held that applying machine learning and data analysis to a particular field of use does not confer eligibility where the claims fail to recite a technical improvement to the underlying computational process itself. Similarly, applying bounding box homography projections and neural networks to track cows in a barn is a field-of-use limitation.
Accordingly, the claims do not integrate the judicial exception into a practical application under Step 2A, Prong Two
Step 2B: (Inventive Concept)
The additional elements, considered both individually and as an ordered combination, do not amount to significantly more than the abstract idea.
The claims recite generic cameras and computer processors performing well-understood, routine, and conventional functions in the computer vision art, including capturing images from overhead, extracting bounding boxes, applying homography projection matrices to adjust for height displacement, stitching bounding boxes, and executing bipartite matching algorithms.
The ordered combination also does not provide an inventive concept. The sequence follows the abstract data-processing steps: capture images from multiple viewpoints, detect bounding boxes, classify standing/lying state, calculate disparate plane projections, stitch the coordinates, and match identities. This is no more than the abstract mathematical idea implemented on generic camera and computer hardware.
Dependent claims 2, 3, 6, and 7 recite selecting a floor level as the stitching plane, utilizing a trained neural network, and applying a bipartite matching algorithm. These limitations merely recite conventional coordinate choices, standard off-the-shelf machine-learning classifiers, and well-known open-source graph-matching algorithms, none of which add significantly more to the underlying abstract concept.
Accordingly, claims 1–3, 6, and 7 are directed to a judicial exception without significantly more and are therefore rejected under 35 U.S.C. § 101.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1–3 and 7 are rejected under 35 U.S.C. §103 as being unpatentable over Ii (Ii et al. (Sept 2021) Distortion Correction and Stitching of Overlapping Cattle Barn Imagez.), in view of Nasirahmadi (Nasirahmadi et al. (2019). Deep Learning and Machine Vision Approaches for Posture Detection of Individual Pigs. In Sensors (Basel, Switzerland) (Vol. 19, Issue 17, p. 3738). PubMed Central.).
Regarding claim 1, Ii teaches an animal husbandry system, wherein a group of animals can move about freely in an area ( §1: stall-free dairy barn and monitoring individual dairy cows throughout the barn. ), the system comprising:
monitoring and analyzing means configured for repeatedly generating images and gathering data therefrom regarding the animals in the area, the monitoring and analyzing means including
( §1, §3.3, §§4.1–4.3: wide-angle cameras on the ceiling yields cost-effective monitoring for extracting, projecting, and matching individual cow regions; generating thirty panoramic images at consecutive times; and generating panoramic video. )
a plurality of cameras provided in such a way that, collectively, the plurality of cameras are suitable for monitoring substantially the complete area from above, and
( §1 and Fig. 1, §4.1, §5: ceiling-mounted cameras provide a bird’s-eye view of the entire barn; twelve ceiling cameras have overlapping fields of view; and the resulting panorama covers the complete barn. )
image processing means, suitable for matching animals which are in a field of view of the plurality of cameras, the image processing means, to this end, being configured for
( §3.3, Fig. 3 and §§4.1–4.2: Mask R-CNN extracts individual cow regions from neighboring camera images, the regions are projectively transformed, and regions (Ma) and (Mb) are matched as the same cow based on IoU. )
detecting at least one silhouette of an animal in a first image of a first camera of the plurality of cameras and in a second image of a second camera of the plurality of cameras,
( §3.3, Fig. 3: individual segmented cow regions are extracted using Mask R-CNN from neighboring source images (a) and (b), corresponding to animal silhouettes detected in first and second camera images. )
defining an animal segmentation mask region around each animal silhouette in each image,
( §3.3, Fig. 3: individual cow regions (Ma) and (Mb) are extracted from neighboring camera images using Mask R-CNN. )
Ii's Mask R-CNN-based extraction discloses an animal segmentation mask region encompassing each detected silhouette but does not itself disclose representing the detected animal by a bounding box. where Nasirahmadi, however, teaches this specific representation:
defining an animal bounding box around each animal silhouette in each image,
( Abstract, §2.1, §2.2.1, Figs. 3 and 7: top-view cameras acquire images containing groups of individual animals; a Faster R-CNN region-proposal network generates a proposed region for each detected animal; each region of interest corresponding to an individual pig is classified; bounding-box regression localizes the coordinates of each detected animal; and Figure 7 shows each individually detected pig enclosed within a bounding box. )
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Ii's system to define an animal bounding box around each Mask R-CNN-generated animal segmentation-mask region, as taught by Nasirahmadi. Nasirahmadi applies bounding-box regression in a neural-network animal detector to localize individual animals in top-view images of group-housed livestock. A bounding box provides a known compact geometric representation of the location and spatial extent of each detected animal and is suitable for subsequent projection, comparison, and association between images. The modification would have amounted to applying a known animal-detection representation to Ii's similar image-based livestock-monitoring system and would have predictably produced an enclosing geometric region for projecting, matching, and identifying the same animal across neighboring camera images.
Then, Ii [as modified by Nasirahmadi] continue to teach:
detecting whether each animal is standing or lying down,
( Nasirahmadi > Abstract, §2.2, Table 1, §§2.2.1–2.2.3, Figs. 2–7, and Table 3: three individual-animal posture classes are defined as standing, lying on the belly, and lying on the side; images containing individual pigs are annotated according to those posture classes; Faster R-CNN, R-FCN, and SSD neural-network object detectors are trained using the annotated images; and the trained detectors classify each detected animal as standing, lying on the belly, or lying on the side. )
applying to each animal bounding box either a first projection onto an image stitching plane in case the animal is standing, or a second projection onto the image stitching plane in case the animal is lying down,
( Ii > [§§3.1 and 3.3 and Figs. 3–4]; Nasirahmadi > [§2.2, §§2.2.1–2.2.3, and Fig. 7]: Ii discloses an ordinary projective transformation M from each source image onto the barn map, which is the floor plane Z=0, and a different height-corrected projective transformation M′ that is applied to standing-cow regions because the back of a standing cow is above the floor; Nasirahmadi classifies each detected animal as standing or lying. In the Ii [as modified by Nasirahmadi]'s system, Nasirahmadi’s posture classification is used to apply Ii’s height-corrected transformation M′ to the bounding box of a standing animal and Ii’s ordinary floor-plane transformation M to the bounding box of a lying animal. )
stitching the projected animal bounding boxes of the first image of the first camera and of the second image of the second camera in the image stitching plane, and
( Ii > [§§3.1–3.3, Fig. 3, and Figs. 5–6]: source images from the ceiling-mounted cameras are projectively transformed onto the common barn-map floor plane and composited to form a panoramic image; cow regions extracted from neighboring source images a and b are distortion-corrected, projectively transformed, and composited onto the empty-barn panoramic image. In the Ii [as modified by Nasirahmadi]'s system, the projected cow regions are represented by their corresponding animal bounding boxes and are stitched in the same common image plane. )
identifying identical animals in the stitched image by comparing the projected animal bounding boxes in the image stitching plane.
( Ii > [§3.3, Fig. 3, and §§4.1–4.2]: projected cow regions Ma and Mb extracted from neighboring camera images a and b are compared using their intersection over union, and Ma and Mb are identified as the same cow when their IoU exceeds a predetermined threshold; the corresponding cow region closest to the image center is retained during compositing to avoid duplication. Ii [as modified by Nasirahmadi]'s system using Nasirahmadi’s bounding-box representation, the same IoU comparison is performed between the projected animal bounding boxes in the common barn-map image plane, thereby identifying detections corresponding to the same animal in the stitched panoramic image. )
Regarding claim 2, Ii [as modified by Nasirahmadi] teaches the system according to claim 1, wherein the image processing means are further configured for using a floor level of the area as the image stitching plane.
( Ii > [§3, §3.1]: the cattle-barn floor map is used as the compositing surface, and the transformation maps each source image to the expressly identified floor plane (Z=0). )
Regarding claim 3, Ii [as modified by Nasirahmadi] teaches the system according to claim 1, wherein the image processing means are further configured for detecting whether each animal is standing or lying down by using an object detection model with neural networks, trained to classify standing and lying animals from images.
( Ii > [§5]; Nasirahmadi > [Abstract, §2.2, Table 1, §§2.2.1–2.2.3, Figs. 2–7, and Tables 2–3]: Ii expressly recognizes the need to determine which cows are standing. Nasirahmadi defines three individual-animal posture classes as standing, lying on the belly, and lying on the side; annotates images of individual animals according to those posture classes; divides the annotated images into training, validation, and testing datasets; and trains neural-network object-detection models, including Faster R-CNN, R-FCN with ResNet101, and SSD with Inception V2, to detect and classify each animal as standing, lying on the belly, or lying on the side. )
Regarding claim 7, the rationales provided in the rejections of claims 1-3 are incorporated herein. Claim 7 recites the identical limitation as claim 3 but depends from claim 2, which further requires using a floor level of the area as the image stitching plane. Because claim 2 depends from and incorporates all the limitations of claim 1, because Ii teaches using the barn-floor plane as the image stitching plane as set forth in the rejection of claim 2, and because Nasirahmadi teaches claim 7’s neural-network posture-classification limitation for the same reasons set forth regarding claim 3, claim 7 is unpatentable over Ii [as modified by Nasirahmadi] for the reasons provided above with respect to claims 1–3.
Claim 6 is rejected under 35 U.S.C. §103 as being unpatentable over Ii [as modified by Nasirahmadi], in view of Nasirahmadi (Nasirahmadi et al. (2019). Deep Learning and Machine Vision Approaches for Posture Detection of Individual Pigs. In Sensors (Basel, Switzerland) (Vol. 19, Issue 17, p. 3738). PubMed Central.).
Regarding claim 6, Ii [as modified by Labrecque] teaches the system according to claim 1,
wherein the image processing means are further configured for identifying identical animals in the stitched image by comparing the projected animal bounding boxes in the image stitching plane using an intersection-over-union (IoU) overlap threshold.
( Ii > [§3.3, Fig. 3, and §§4.1-4.2], Nasirahmadi > [§2.1, §2.2.1, Figs. 3 and 7]: Ii projectively transforms animal regions from neighboring camera images and identifies regions Ma and Mb as representing the same cow when their IoU exceeds a predetermined threshold; Nasirahmadi supplies the bounding-box representation for the detected animal regions. )
Ii [as modified by Nasirahmadi] teaches identifying identical animals by comparing projected animal bounding boxes in the image-stitching plane but does not expressly teach performing the comparison using a bipartite matching algorithm. Cai, however, teaches this feature:
wherein the image processing means are further configured for identifying identical objects in the image by comparing the projected object bounding boxes in the image plane using a bipartite matching algorithm.
( Cai > [Abstract, §3.2, §§4.2–4.3, §5, §9.2.1, Fig. 4]: object detections in synchronized images from different camera views are represented by bounding boxes, and bounding boxes representing the same object instance are associated across the images; bounding-box locations from the respective camera views are homographically projected into a common main-camera image plane, specifically a top or bird’s-eye view, by projecting representative points derived from the bounding boxes; the projected locations are compared using a pixel-distance cost matrix; and the Kuhn–Munkres, also known as the Hungarian, algorithm performs maximum bipartite matching with minimum loss. The resulting assignment matrix indicates a match or non-match between bounding box i in image 1 and bounding box j in image 2, with each bounding box being matched to no more than one bounding box in the other image. )
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the matching process of Ii [as modified by Nasirahmadi] to use Cai’s bipartite matching algorithm when comparing the projected animal bounding boxes. Ii [as modified by Nasirahmadi] and Cai address the same technical problem of determining correspondence between multiple detections of the same physical object obtained from different cameras, and both geometrically relate the detections in a common image plane. Cai’s use of projected bounding-box-derived locations and a cross-image distance cost matrix therefore would have been directly compatible with Ii [as modified by Nasirahmadi]’s comparison of projected animal regions. The modification would have amounted to applying a known global object-assignment technique to a similar multi-camera matching system and would have predictably provided mutually consistent one-to-one correspondences while reducing conflicting or duplicate animal matches. Further, Ii [as modified by Nasirahmadi] expressly recognizes duplicated and missing cows as matching errors and identifies improvement of the matching process as a desired objective (§5).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEN KUDO whose telephone number is (571)272-4498. The examiner can normally be reached M-F 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
KEN KUDO
Examiner
Art Unit 2671
/KEN KUDO/Examiner, Art Unit 2671
/VINCENT RUDOLPH/Supervisory Patent Examiner, Art Unit 2671
Prosecution Insights