DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/13/2026 has been entered.
Response to Arguments
Applicant’s response to the last Office Action, filed 4/13/2026, has been entered and made of record.
Applicant has amended claims 1-4, 11, 13-15, and 17-20. Claims 5 and 16 are cancelled. Claims 1-4, 6-15, 17-22 are currently pending.
Applicant’s arguments, filed 4/13/2026, with respect to the rejection of claim 1, 11, and 20 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn. However, upon further consideration, a new ground of rejection is made in view of Ali (U.S. Patent Pub. No. 2023/0282122).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 11-12, and 20-22 are rejected under 35 U.S.C. 103 as being unpatentable over Kriesel (U.S. Patent Pub. No. 2005/0257748) in view of Ali (U.S. Patent Pub. No. 2023/0282122).
Regarding Claim 1, Kriesel teaches a method for automated evaluation of animals, comprising:
- accessing sensor data acquired of an animal body, wherein the sensor data comprises at least two different sensor data types associated with evaluating one or more target traits of the animal body (¶180 This invention may be used to acquire physical dimensions of cattle or hogs as they pass through a common chute as shown in FIG. 2-1. In this example, range cameras with illuminators are located on three sides of the target animal. An infrared camera is also positioned over the animal to obtain thermal images of the back region;)
- applying feature extraction using at least one machine learning model applied to derivative sensor data generated by combining together different sensor types (¶236 The processing channel from visible spectrum cameras to measurement tables and display is shown in greater detail in FIG. 2-17. In this Figure, the target animal is illuminated with structured illumination and images are obtained from three strategic camera positions (additional or fewer positions may be required for a shaper of different complexity)... The 3D merger algorithm is then used to align, register and combine the independent view data sets into one unified data set; ¶237 FIG. 2-18 shows the same processing channel as FIG. 2-17 only with the addition of a thermal imaging camera for obtaining thermal images,) to extract trait-specific feature data associated with the one or more target traits; and (¶256 teaches determining backfat; ¶260 2.11 Measurement Techniques shows many measurement options for the data, like calculating volume ¶262, truncated plane volumes ¶264, hip height ¶269, hip width ¶271, and many more.)
- generating one or more evaluation scores for the animal body, for each of the one or more target traits, based on the extracted trait-specific feature data for these target traits (¶302 With the complete 3D data set available from this invention and the numerous volumetric and dimensional measurements which can be computed from that data set, a number of statistical analyses are possible and advantageous; ¶720 3.7 Application of Measurements to Livestock Evaluations; ¶730 The measurements of this invention may be used to automate the grading of cattle and hogs. FIG. 3-16 is a chart showing U.S. Quality Grades of prime, choice, select, standard, and utility. From a simple visual comparison of prime and utility grades it is evident that measurements such as hip width, hip height, and volume can easily discriminate between the extremes of prime and utility grades. The measurements of this present invention make the discrimination between the other grades equally clear by comparing a set of measurements to an empirically determined set of standard grading measurements which are characteristic of each of the U.S. quality grades.)
wherein the evaluation scores reflect one or more of quantitative and qualitative traits of the animal body, and the evaluation is performable in both the presence and absence of markers on the animal body (¶183 The silhouette or profile data only provides measurement potential around the outline of the silhouette since no landmarks exist within the darkened shape; ¶184 The 2D video image has the volumetric limitations of the silhouette data with no ability to account for surface concavities. Though the 2D video data does provide the ability to locate landmarks within the 2D silhouette outline, all surface features and measurements are obtained as their corresponding projections onto a flat, 2D surface.)
Kriesel does not explicitly disclose applying feature extraction using at least one machine learning model applied to derivative sensor data generated by combining together different sensor types.
Ali is in the same field of art of image analysis. Further, Ali teaches applying feature extraction using at least one machine learning model applied to derivative sensor data generated by combining together different sensor types (¶88 sensor data from these sensors may be combined to further enhance an image of the first object 560. For example, an image from a camera may be combined with imagery generated from LIDAR sensor data and imagery generated from SONAR sensor data; Feature detection and extraction techniques may be applied to the image to obtain a set of features useful in identifying the first object 560. In some examples, convolution neural networks, support vector machines, and/or deep learning methods are used to extract features of the object and/or identify the object.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kriesel by extracting features of the derivative data that is taught by Ali; thus, one of ordinary skilled in the art would be motivated to combine the references to improve image resolution for feature detection and extraction (Ali ¶88).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 2, Kriesel in view of Ali discloses the method of claim 1, wherein the sensor data types comprise two or more of:
- the 2D image data generated by one or more 2D imaging sensors;
- the depth sensor data generated by one or more depth sensors;
- the infrared (IR) sensor data generated by one or more IR sensors;
- the depth sensor data generated by applying monocular depth estimation to the two- dimensional (2D) image data; and
-non visual sensor data types including ultrasound data, weight data, or other physiological or health sensing data (Kriesel, ¶180 This invention may be used to acquire physical dimensions of cattle or hogs as they pass through a common chute as shown in FIG. 2-1. In this example, range cameras with illuminators are located on three sides of the target animal. An infrared camera is also positioned over the animal to obtain thermal images of the back region;) (Ali, ¶88 sensor data from these sensors may be combined to further enhance an image of the first object 560. For example, an image from a camera may be combined with imagery generated from LIDAR sensor data and imagery generated from SONAR sensor data.)
Regarding claim 11, claim 11 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Kriesel further teaching on: system for automated evaluation of animals, comprising:
- an evaluation apparatus comprising one or more sensors, and at least one first processor coupled to the one or more sensors (¶610 A control unit takes as input, the signals from the proximity sensors that identify when the target animal is within the target zone. This control unit, in turn, outputs trigger signals to initiate image capture procedures by the range and thermal imaging cameras. This control unit may consist of discrete digital circuitry, digital and analog circuitry, microprocessor-based circuitry. The control function may also be combined with the image processing function within the processing unit; ¶612 The processing unit implements the algorithms, image processing, surface processing, volume processing, and measurements described within the Summary section of this application. Digital signal processing (DSP) components from such companies and Texas Instruments and Analog Devices Inc. are prime candidates for inclusion in this unit. Additionally, array processing subsystems and boards may be used to increase the processing speed if desired.)
Regarding Claim 12, Kriesel in view of Ali discloses the system of claim 11, wherein the evaluation apparatus comprises one or more of an automated evaluation assembly (AEA), and a user device (Kriesel, ¶335 A key component to this invention is a convenient, useful, user interface.)
Regarding claim 20, claim 20 has been analyzed with regard to claim 1 and is rejected for the same reasons of obviousness as used above as well as in accordance with Ali further teaching on: An evaluation apparatus for evaluating animals comprising:
- one or more sensors for generating sensor data; and - at least one non-transitory memory storing computer executable instructions, which when executed by at least one processor cause the at least one processor to execute the method (¶171 The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention; ¶172 A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se)
Regarding Claim 21, Kriesel in view of Ali discloses the method of claim 1, wherein the animal body comprises an animal carcass (Kriesel, ¶179 The present invention has been conceived to obtain volumetric, curvilinear and linear measurements of livestock animals and full carcasses, specifically cattle, and hogs.)
Regarding Claim 22, Kriesel in view of Ali discloses the method of claim 11, wherein the animal body comprises an animal carcass (Kriesel, ¶179 The present invention has been conceived to obtain volumetric, curvilinear and linear measurements of livestock animals and full carcasses, specifically cattle, and hogs.)
Claims 3, 10, and 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Kriesel (U.S. Patent Pub. No. 2005/0257748) in view of Ali (U.S. Patent Pub. No. 2023/0282122) in view of Psota (U.S. Patent Pub. No. 2024/0382109).
Regarding Claim 3, Kriesel in view of Ali teaches the method of claim 2.
Kriesel in view of Ali does not explicitly disclose generating the derivative sensor data by applying one or more of:
- an object detection machine learning model to the 2D image data to generate an object annotated 2D image with indicia of the location of the animal in the 2D image;
- a body part detection machine learning model to the object annotated 2D image to generate a body part annotated 2D image, the body part annotated 2D image comprising indicia of the locations of different animal body parts; and
- a landmark detection machine learning model to the body part annotated 2D image to generate landmark data.
Psota is in the same field of art of livestock image analysis. Further, Psota teaches generating the derivative sensor data by applying one or more of:
- an object detection machine learning model to the 2D image data to generate an object annotated 2D image with indicia of the location of the animal in the 2D image;
- a body part detection machine learning model to the object annotated 2D image to generate a body part annotated 2D image, the body part annotated 2D image comprising indicia of the locations of different animal body parts; and
- a landmark detection machine learning model to the body part annotated 2D image to generate landmark data (¶81 a process is started using the body part detection network (fully convolutional neural network) to look for a pig of interest to enter the frame. Once the pig enters, its body center is tracked across the frame until it exits the field of view. This encapsulates a walking event video with associated with a read ID from a tag associated with the animal. To detect joints or anatomical landmarks in a walking event for a pig of interest, each frame of the trimmed video or set of images is processed with a deep joint detection network to detect the nose, mid-section, tail, and leg joints of interest. In some embodiments, a YOLOv3 object detection model is applied to isolate animals, such as gilts, from the background image.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kriesel in view of Ali by using trained object detecting to generate landmark data that is taught by Psota; thus, one of ordinary skilled in the art would be motivated to combine the references for accurate phenotypic measurements of the livestock (Psota ¶4).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 10, Kriesel in view of Ali in view of Psota discloses the method of claim 1, wherein the one or more evaluation scores are stored in association with an animal profile (Psota, ¶103 Using tag-based identifiers or other identification means, such as a machine-vision system, to individually identify each animal 130 that traverses the walkway 122 provides for the system 100 to individually provide gait patterns, phenotype predictions or determinations, or health outcomes or predictions for each individual animal,) and the evaluation scores are output on a display interface of a user device (Psota, ¶96 The display 40 is in electronic communication with the application server 11 and may provide for the viewing of a GUI displaying predicted phenotypic information or health predictions or outcomes.)
The reasons for combining Krisel and Ali and Psota are similar to that stated in the rejection of claim 3. In addition, this same reasoning is pertinent and applicable to the rejections of claim 13 below.
Regarding Claim 13, Kriesel in view of Ali in view of Psota discloses the system of claim 12, wherein the at least one first processor of the evaluation apparatus is included in a controller of the AEA, and the AEA further comprises: (i) a frame structure for supporting the one or more sensors; and (ii) an area for receiving the animal being evaluated, and (Kriesel, Fig. 2-1; ¶651 As indicated in FIG. 2-1, the data processing means of the invention for processing image data from the cameras 22, 24, 26, and 28 with which it is operably associated, here comprises a control and processing unit 42. Unit 42 is supported on a table 52 that is preferably located adjacent the animal positioning device 12. Personal computer 44 and an associated monitor 53 are also preferably located on table 52; ¶652 FIG. 3-2 shows an alternate form of the equipment layout. This latest embodiment is similar in many respects to that shown in FIG. 2-1 and like numerals are used to designate like components. Here the apparatus includes three range cameras 27 and three IR cameras 28. Target visibility is enhanced from the side views by replacing the normal chute bars with blackened, steel cables 33 in the regions viewed by the cameras. The blackened cables are nearly invisible to the mesh processing algorithm and thermal cameras)
wherein the user device hosts a mobile application which is executed by the at least one processor (Psota, ¶96 the display 40 is associated with a separate computer or computing device, such as a smartphone, tablet, laptop, or desktop computer which is used by a user to remotely view and access the application server 11,) the mobile application being configured to operate the one or more sensors, and transmit the sensor data to the at least one second processor (Psota, ¶91 The sensors 30 through 30n comprise a set of sensors connected to the application server 11 through electronic communications means, such as by Ethernet or BLUETOOTH connections. The sensors 30 through 30n may comprise sensors such as image sensors (e.g., electronic video cameras or CCD cameras), RFID readers, pressure sensors, weight sensors, or proximity sensors,) the mobile application also being configured to receive the evaluation scores from the at least one second processor and display the evaluation scores on a display interface of the user device (Psota, ¶89 The weight estimates and other phenotypic information or predictions are provided to a user through a locally accessible or web-based graphical user interface (“GUI”).)
Claim 14 recites limitations similar to claim 3 and is rejected for the same rationale and reasons of obviousness as used above.
Claims 7-9 and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over Kriesel (U.S. Patent Pub. No. 2005/0257748) in view of Ali (U.S. Patent Pub. No. 2023/0282122) in view of Morris (U.S. Patent Pub. No. 2023/0206469).
Regarding Claim 7, Kriesel in view of Ali teaches the method of claim 2, wherein generating the derivative sensor data comprises generating derivative depth sensor data by:
- optionally, converting depth sensor data of the animal into point cloud data (Kriesel, ¶138 There are many methods for visualization of volume data… Both methods may begin with a 3D point cloud of data points as might be obtained from one or more range images;)
- applying 3D coordinate registration to the point cloud data to generate registered point cloud data (Kriesel, ¶236 Each of the digital images are processed by the range image algorithm to obtain a three-dimensional point cloud set. These points may be oriented somewhat arbitrarily with respect to a given coordinate system;)
- using the 3D coordinate registered data to generate a 3D model reconstruction of the animal; and (Kriesel, ¶236 The mesh algorithm is applied to each view data set to convert the arbitrarily-spaced point cloud data into a mesh surface with a grid coordinate system. The mesh algorithm is described hereinafter and is illustrated in FIG. 2-19, and in FIGS. 2-20A through 2-20H. Upon application of the mesh algorithm to each of the three different view data sets, the data takes the form shown in FIG. 2-21. In FIG. 2-21 the three, 3D views of the target animal exist separately but with the same coordinate system.)
Kriesel in view of Ali does not explicitly disclose applying 2D to 3D landmark projection to generate 3D landmark data.
Morris is in the same field of art of livestock image analysis. Further, Morris teaches applying 2D to 3D landmark projection to generate 3D landmark data (Morris, ¶42 a Transpositional Tagging technique can provide advantages including but not limited to: ¶43 detecting features in one type of image (such as IR images) in which the feature is more apparent (whether because it is easier for a human tasked with tagging features in an image to see, because it is only evident in certain types of images, or otherwise), and then transferring the location of that feature to a type of image that has greater information relevant to full 3D motion, such as a 3D depth image.)
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Kriesel in view of Ali by applying 2D to 3D landmark projection that is taught by Morris; thus, one of ordinary skilled in the art would be motivated to combine the references to quickly generate high accuracy pig landmark annotations for depth-based pose estimation (Morris ¶39).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 8, Kriesel in view of Ali in view of Psota discloses the method of claim 7, further comprising applying 3D feature extraction based on one or more of: (i) the registered point cloud data; (ii) the reconstructed 3D model data; and (iii) the 3D landmark data, to extract one or more features related to the one or more target traits (Kriesel, ¶261 Volumetric and Dimensional Measurements Calculated Directly from 3D Data; see claim 1.)
Regarding Claim 9, Kriesel in view of Ali in view of Psota discloses the method of claim 2, further comprising:
applying IR pixel mapping calibration between the IR data and the 2D image to generate IR pixel mapped image data; and applying IR feature extraction to one or more of: (i) IR data; and (ii) IR pixel mapped data (Morris, ¶41 In the example of livestock detection, markers were used that were visible in infrared images associated with IR depth data acquisition. As described below, these markers can be semi-automatically detected and tracked, and then their positions transposed to the depth images/video frames output by the depth IR camera at this same moment in time. The resulting depth images, now tagged with various structural locations relevant for pose, posture, and/or movement estimation were determined to be useful for modeling 3D motion of animals and, in turn, for training neural networks to make various motion-related predictions and detections.)
The reasons for combining Krisel and Ali and Psota are similar to that stated in the rejection of claim 7.
Claim 17 recites limitations similar to claim 7 and is rejected under the same rationale and reasoning.
Claim 18 recites limitations similar to claim 8 and is rejected under the same rationale and reasoning.
Claim 19 recites limitations similar to claim 9 and is rejected under the same rationale and reasoning.
Allowable Subject Matter
Claims 4, 6, and 15 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claims 4 and 15, no prior art teaches wherein the applying the landmark detection machine learning model comprises: - applying a trained backbone network which receives the 2D image data and extracts one or more features relevant to landmark detection; - applying a trained head network comprising a convolutional neural network (CNN), wherein the CNN receives the extracted features and the 2D image, and identifies indicia corresponding to candidate landmarks,
wherein the CNN applies region-wide landmark detection using three sizes of masks, and further, determines a confidence score for each detected candidate landmark detected in each mask size; and
- selecting, from the candidate landmarks, the landmark with highest score for each instance of the region-wise landmark detection.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DUSTIN BILODEAU whose telephone number is (571)272-1032. The examiner can normally be reached 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at (571) 272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DUSTIN BILODEAU/Examiner, Art Unit 2664
/JENNIFER MEHMOOD/ Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights