DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner’s Note
The instant application has a lengthy prosecution history and the examiner encourages the applicant to have an interview (telephonic or personal) with the examiner prior to filing a response to the instant office action. Also, prior to the interview the examiner encourages the applicant to present multiple possible claim amendments, so as to enable the examiner to identify claim amendments that will advance prosecution in a meaningful manner.
Continued Examination Under 37 CFR 1.114
The present application is being examined under the pre-AIA first to invent provisions. A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 1/16/2026 has been entered.
Response to Arguments/Amendments
Presented arguments have been fully considered, but some are rendered moot in view of the new ground(s) of rejection necessitated by amendment(s) initiated by the applicant(s). Examiner fully addresses below any arguments that were not rendered moot.
Claim Rejections - 35 USC § 103
Summary of Arguments:
Regarding claims 1, 2, 5, 9, 10, 11, 15 and 18 Applicant argues that the combined references of Ivan, Marshall, Fausto, Thomas and Sung fail to teach or suggest at least the features of "wherein... counts a number of animal entities by identifying the animal entities in the image-capturing region for the image data to which the calibration is applied... wherein the camera is fixed on an upper portion of the straight passage" and "wherein the animal entity status information generation program identifies the animal entity when the animal entity enters the image-capturing region based on characteristic information of the animal entity extracted from the image data or a color change of pixels in the image data, and counts the animal entity" in currently amended claim 1. Applicant respectfully asserts that Marshall does not disclose the feature of counting animals using image data captured by a camera, as claimed. As clearly set forth in the reference, the counting of animals is performed using a barcode scanning method or RFID-based scanning, not by analyzing images captured by a camera, as claimed. The camera disclosed in Marshall is not used to count animals, but rather is used solely for three-dimensional modeling or reconstruction of individual animals. In other words, Marshall explicitly relies on non-image-based identification mechanisms, such as barcode tags or RFID tags attached to animals, in order to perform the counting. Accordingly, Marshall fails to teach or suggest the feature of counting animal individually based on an image analysis or the feature of detecting animal entry into an imaging region and incrementing a count based on visual features, as claimed. Neither Ivan nor Marshall, alone or in combination, discloses or suggests the features of detecting an animal entering an imaging region of a straight passage, identifying individual animals based on image features or pixel-level color changes, and incrementing an animal count solely based on image analysis, without auxiliary identification hardware
Examiner’s Response:
Examiner respectfully disagrees. Regarding claims 1, 2, 5, 9, 10, 11, 15 and 18, Examiner contends the claim limitation “counts a number of animal entities by identifying the animal entities in the image-capturing region for the image data to which the calibration is applied” does not include “counting animals using image data captured by a camera”. The claim portion does not describe how the “identifying the animal entities in the image-capturing region for the image data to which the calibration is applied” in order to count a “number of animal entities”. The claim does not recite any kind of “analyzing images captured by a camera” in order to count the “number of animal entities”.
The claim portion “wherein the animal entity status information generation program identifies the animal entity when the animal entity enters the image-capturing region based on characteristic information of the animal entity extracted from the image data or a color change of pixels in the image data, and counts the animal entity” seems to suggest that
the animal entity status information generation program:
1) identifies the animal entity when the animal entity enters the image-capturing region based on characteristic information of the animal entity extracted from the image data or a color change of pixels in the image data, and
2) counts the animal entity.
This claim limitation at best describes “the animal entity status information generation program” performing two steps (i.e. (1) and (2)) that, from the way they are claimed, can be done separately. Furthermore, it is not clear whether the “image data” mentioned in “counts a number of animal entities by identifying the animal entities in the image-capturing region for the image data to which the calibration is applied” are the same “image data” mentioned in “wherein the animal entity status information generation program identifies the animal entity when the animal entity enters the image-capturing region based on characteristic information of the animal entity extracted from the image data or a color change of pixels in the image data”; let alone if “identifies the animal entity when the animal entity enters the image-capturing region based on characteristic information of the animal entity extracted from the image data or a color change of pixels in the image data”= “by identifying the animal entities in the image-capturing region for the image data to which the calibration is applied”.
In response to applicant’s argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., counting animals using image data captured by a camera or analyzing images captured by a camera) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
Furthermore, Ivan teaches identifying livestock using images (i.e. acquiring, by a two-dimensional camera (7), in a scene, a two dimensional image of at least one object (200); identifying the object (200) within the acquired two dimensional image- abstract, fig. 2…) and Marshall teaches identifying animal using image (i.e. This color, still image 520 of the target animal may be used for future identification of the animal and visual information such as color or breed…[1391] b) an animal identification (I.D.) number or code…[1405] d) automatic entry from electromagnetic or thermal scanning of an animal I.D. number or code located somewhere on the target animal- ¶1389-1405).
In response to applicant’s arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986).
Accordingly, Examiner maintains the rejections.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1, 2, 5-9 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites the limitation “the image data to which the calibration is applied” and “the image data” in lines 10 and the second to last line, respectively. There are insufficient antecedent basis for these limitations in the claim. For the “image data” in the last limitations, it is not clear whether the antecedent basis is “image data to which the calibration is applied” or “the image data obtained by correcting the image distortion”.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 2, 5, 9, 10, 11, 15 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Ivan Amat Roldan et al. [US 10249054 B2: already of record] in view of Marshall S. Kriesel et al. [US 20050257748 A1: already of record] further in view of Fausto Bernardini et al. [US 20040217260 A1: already of record].
Regarding claim 1, Ivan teaches:
1. An animal entity status information providing device based on an image (i.e. The present invention relates to a method and to a device for automatically calculating parameters of an object, for instance the size, dimensions, body part dimensions, body features, volume, weight and other related data allowing defining said object, which preferably would be an animal such as a livestock animal- Col 1, line 15-21), the animal entity status information providing device comprising:
a memory configured to store an animal entity status information generation program (i.e. In accordance with yet another embodiment, and as seen in FIG. 9, once the two dimensional image (or even the sequence of images and/or the video sequence) is acquired by a user via a computing software application installed the proposed device 100 (step 91), the acquired image is transmitted to a processing unit located in a remote server S of a cloud computing environment in order a computing algorithm running in the remote server S (step 92) validating and processing the received image (in few seconds) enabling in that way additional web services like pig growth monitoring in an ubiquitous manner or building of production maps. Once the data has been processed (step 93) it is transmitted to the proposed device 100 allowing said computing software application displaying (step 94) the calculated value of the parameter- col 8, line 19-33); and a processor configured to execute the animal entity status information generation program, wherein as the animal entity status information generation program is executed by the processor (i.e. Depending on the model complexity the first and/or second means to calculate the object or objects parameters can be, entirely or in part, included in the proposed device or in a remote server located in a cloud computing environment- col 5, line 47-51),
the animal entity status information generation program performs calibration on a camera that sets a region of through which an animal entity passes as an image-capturing region according to an initialization operation, sets a scale for an actual length per pixel by using image data received from the camera (i.e. According to this embodiment, the device 100 is placed at a known distance d (that is, a known situation in the scene) and a size of the pixel of the object 200 is already known (previously calculated, step 20, according to the teachings of the invention)- col 7, line 29-33… In this case, a calibration object OB of a known size, like a green panel of 30 by 30 cm, is placed next to the pig 200. In addition, the pig 200 may be positioned at a known position (a priori knowledge of the scene), e.g. by knowing the size of the batch SB or by knowing the size of a grid or tiles on the floor SG. This enables to precisely calculate parameters of the pig 200 by: segmenting the pig 200 and the calibration object OB based on a color appearance segmentation algorithm; affine registration of the calibration object OB to calculate a spatial transformation and the distance between the device 100 and the pig 200; calculating the pig volume based on a 3D template and a spatial transformation; and calculate the pig weight based on an heuristic model calibrated with experimental data- col 7, line 51-64),
calculates sizes of the animal entities in the image-capturing region by using the scale, estimates weights of the animal entities based on the sizes of the animal entities (i.e. the size of a pixel of the object in the acquired and segmented two dimensional image taking into account the distance between the object and the two-dimensional camera; and calculating, by a second means, several parameters of the object including at least the size, dimensions, body part dimensions, body features (including a body fat index), weight and/or volume by using said calculated size of the pixel and an a priori model of the object- col 3, line 27-34), and outputs status information on the animal entities including the number of animal entities and the estimated weights of the animal entities (i.e. Once the data has been processed (step 93) it is transmitted to the proposed device 100 allowing said computing software application displaying (step 94) the calculated value of the parameter- col 8, line 30-33),
wherein the animal entity status information generation program identifies a corresponding animal entity when identifying that the animal entity enters the image-capturing region, identifies the animal entities based on characteristic information of the animal entity extracted from the image data or a color change of pixels in the image data (i.e. The method comprising: acquiring, by a two-dimensional camera (7), in a scene, a two dimensional image of at least one object (200); identifying the object (200) within the acquired two dimensional image; calculating, by a first means, the size of a pixel of the object (200) in the acquired and segmented two dimensional image taking into account the distance between the object (200) and the two-dimensional camera (7); and calculating, by a second means, several parameters including at least the size, dimensions, body part dimensions, body features, weight and/or volume of the object (200) by using said calculated size of the pixel and an a priori model of the object (200), wherein said a priori model includes information linking different parts, contours or shapes representative of several objects (200), previously acquired with a two-dimensional camera, with several parameters said several objects (200).- Abstract).
However, Ivan does not teach explicitly:
a straight passage; counts a number of animal entities by identifying the animal entities in the image-capturing region for the image data to which the calibration is applied;
wherein the camera is fixed on an upper portion of the straight passage, counts an identified animal entity.
In the same field of endeavor, Marshall teaches:
a straight passage (i.e. see fig. 2-14… From a system perspective it may be possible to reduce the speed requirements and cost of the camera systems by simply designing a different chute system which slows the animals down as they pass through the camera area- ¶0221); counts a number of animal entities by identifying the animal entities in the image-capturing region for the image data to which the calibration is applied (i.e. This color, still image 520 of the target animal may be used for future identification of the animal and visual information such as color or breed. The input data region of this page also includes additional information about the target animal such as…b) an animal identification (I.D.) number code- ¶1389-1391… automatic entry from electromagnetic or thermal scanning of an animal I.D. number or code located somewhere on the target animal- ¶1405…The number of animals scanned is counted in the lower left of the page (704). The count is displayed as both a truck count and a cumulative count for the given owner- ¶1432… Since the automated data acquisition may be triggered by either analysis of a live video camera or the output of a proximity detector, the trigger technology is identified by mouse clicking over the appropriate box in region (972). Similarly, the trigger mode (973) and the I.D. method (974) may also be selected- ¶1462);
wherein the camera is fixed on an upper portion of the straight passage (i.e. As indicated in FIG. 2-1, the data processing means of the invention for processing image data from the cameras 22, 24, 26, and 28- ¶0651),
counts an identified animal entity (i.e. The method comprising: acquiring, by a two-dimensional camera (7), in a scene, a two dimensional image of at least one object (200); identifying the object (200) within the acquired two dimensional image; calculating, by a first means, the size of a pixel of the object (200) in the acquired and segmented two dimensional image taking into account the distance between the object (200) and the two-dimensional camera (7); and calculating, by a second means, several parameters including at least the size, dimensions, body part dimensions, body features, weight and/or volume of the object (200) by using said calculated size of the pixel and an a priori model of the object (200), wherein said a priori model includes information linking different parts, contours or shapes representative of several objects (200), previously acquired with a two-dimensional camera, with several parameters said several objects (200)- Abstract).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan with the teachings of Marshall accurate and convenient methods to evaluate fat content or lean:fat ratios (Marshall- ¶0167).
However, Ivan and Marshall do not teach explicitly:
wherein the animal status information generation program corrects image distortion based on an image obtained by capturing a checkerboard placed in the one region of the straight passage sets the scale, and calculates the scale for the actual length per a pixel by comparing a length of a unit line segment of the checkerboard with a number of pixels representing the unit line segment of the checkerboard in the image data obtained by correcting the image distortion.
In the same field of endeavor, Fausto teaches:
wherein the animal status information generation program corrects image distortion based on an image obtained by capturing a checkerboard placed in the one region of the straight passage and set the scale, and calculates a scale for an actual length to a pixel by comparing a length of a unit line segment of the checkerboard with a number of pixels representing the unit line segment of the checkerboard in image data obtained by correcting image distortion (i.e. The calibration begins with the acquisition of the cube in a standard position, as shown in FIG. 12. The corners of the three checkerboards are then detected for providing camera 100 calibration data. The preferred camera 100 calibration method employs several fully automatic phases during which patterns of increasing complexity are detected. But, depending on the visual feedback, the user decides at which phase to begin and provides input clicks accordingly. The full process is applied independently to each checkerboard. Each phase begins with a known pattern defined by a certain number of points. In phase 1, the user clicks a triangle (a, b, c) representing a normal basis of the checkerboard. Then, from the extrapolated parallelogram (a, b, d, c), and using an automatic algorithm, each of the four corners is automatically adjusted so that the quadrangle matches "perfectly" the corresponding image edges. This is done by maximizing the integral of the magnitude of the image intensity gradient along the edges of the quadrangle pattern. The integral can be maximized by using a gradient ascent algorithm in multiple image scales to avoid irrelevant local minima. The initial parallelogram is allowed to deform as a general quadrangle because the camera 100 projection does not preserve parallelism. Each additional phase starts either using the known corners resulting from the previous phase or, if the user selects, by the same corners but specified manually. During the second phase the known quadrangle is extended by linear extrapolation to form the quadrangle (a, e, f, g). Using the same optimization algorithm, these four corners are then fitted automatically to the image data. The corner h is then initialized by quadratic extrapolation using a, b and e, accounting to some extent for projective depth distortion. The corners i and j are initialized similarly and the quadrangle (a, h, i, j) is automatically fitted. This process is repeated until all the corners (a, b, e, h, . . . , k), (a, d, f, i, . . . , l) and (a, c, g, j, . . . , m) are found. During the third phase, using linear, and whenever possible quadratic extrapolations, followed by automatic fitting to image data, eventually all the corners of the boundary of the checkerboard are detected. It is possible to do so by only considering a quadrangle with edges at least as large as half of the full checkerboard side. Avoiding shorter edges has been found to increase robustness. During the fourth phase, and starting from the known boundary corners, all the inside corners are initialized by simply intersecting "horizontal" and "vertical" lines. Due to camera 100 lens radial distortion, the "lines" are actually slightly curved, and deviations from the ideal line on the order of three pixels can be observed. This is accounted for during the fifth phase by fitting individual cross-hair patterns for each corner. The cross-hair pattern (s, t, u, v) is initialized and automatically fitted. The only allowed degrees of freedom during the optimization are the translation and the angle between su and tv. Furthermore, for each cross-hair pattern, this algorithm is repeated a number of times, starting from randomly selected nearby initial guesses. The corresponding corner is only considered as successfully detected if all of these tries yield the same result, up to some fraction e.g., one tenth, of a pixel. This is one suitable technique to enforce the significance of the detected corner. Finally, the detected corners are displayed, superimposed with the checkerboard image. The user then accepts or rejects the achieved detection, thereby completing the camera calibration procedure- ¶0033).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan and Marshall with the teachings of Fausto to achieve sub-millimeter accuracy so that when scans are registered and combined with the colors, the fine-detailed normals are not blurred (Fausto- ¶0032).
Regarding claim 2, Ivan, Marshall and Fausto teach all the limitations of claim 1 and Ivan further teaches:
further comprising:
a communication module configured to receive the image data from the camera (i.e. the proposed device 100 requires a two-dimensional camera 7 that preferably will have connectivity to internet, WiFi or some sort of mechanism to transmit data to a processing unit comprising several means which can be included, entirely or in part, on the same device 100 or in a remote server S- col 7, line 24-28), wherein the communication module transmits the number of the animal entities and the estimated weights of the animal entities to one of a user terminal and an external computing device (i.e. In accordance with yet another embodiment, and as seen in FIG. 9, once the two dimensional image (or even the sequence of images and/or the video sequence) is acquired by a user via a computing software application installed the proposed device 100 (step 91), the acquired image is transmitted to a processing unit located in a remote server S of a cloud computing environment in order a computing algorithm running in the remote server S (step 92) validating and processing the received image (in few seconds) enabling in that way additional web services like pig growth monitoring in an ubiquitous manner or building of production maps. Once the data has been processed (step 93) it is transmitted to the proposed device 100 allowing said computing software application displaying (step 94) the calculated value of the parameter- col 8, line 19-33).
Regarding claim 5, Ivan, Marshall and Fausto teach all the limitations of claim 1.
However, Ivan does not teach explicitly:
wherein the animal entity status information generation program automatically performs an operation of counting the number of the animal entities and estimating the weights of the animal entities from first time when the animal entities are identified in the image-capturing region to second time when the animal entities are not identified in the image-capturing region.
In the same field of endeavor, Marshall teaches:
wherein the animal entity status information generation program automatically performs an operation of counting the number of the animal entities and estimating the weights of the animal entities from first time when the animal entities are identified in the image-capturing region to second time when the animal entities are not identified in the image-capturing region (i.e. The large number of animals necessitates an automated measurement system which acquires, processes and records the measurement data rapidly. In a slaughter plant situation, an animal may be slaughtered every 3 to 10 seconds. A lengthy measurement process is not acceptable. Additionally, live animals are often moving. Even carcasses are in constant motion on an overhead conveyor belt. To achieve an accurate measurement an apparatus must be capable of freezing such movement. The technologies represented in FIGS. 1-1 and 1-2 need to be examined in light of light of the specific requirements for measuring live and carcass cattle and hogs- ¶0053).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan with the teachings of Marshall accurate and convenient methods to evaluate fat content or lean:fat ratios (Marshall- ¶0167).
Regarding claim 9, Ivan, Marshall and Fausto teach all the limitations of claim 1.
However, Ivan does not teach explicitly:
wherein the animal entity status information generation program further outputs a suitable grade, an under grade, or an exceeding grade for each animal entity which are classified according to a difference between a reference weight and the estimated weights as animal entity status information and as a result of comparing the estimated weight with a preset reference weight or a reference weights set by a user.
In the same field of endeavor, Marshall teaches:
wherein the animal entity status information generation program further outputs a suitable grade, an under grade, or an exceeding grade for each animal entity which are classified according to a difference between the reference weight and the estimated weight as animal entity status information and as a result of comparing the estimated weight with a preset reference weight or a reference weight set by a user (i.e. 3.8 Automated Grading… The measurements of this invention may be used to automate the grading of cattle and hogs. FIG. 3-16 is a chart showing U.S. Quality Grades of prime, choice, select, standard, and utility. From a simple visual comparison of prime and utility grades it is evident that measurements such as hip width, hip height, and volume can easily discriminate between the extremes of prime and utility grades. The measurements of this present invention make the discrimination between the other grades equally clear by comparing a set of measurements to an empirically determined set of standard grading measurements which are characteristic of each of the U.S. quality grades. One method for automating the grading of cattle and hogs uses n normalized measurements in a measurement space. The normalization of each measurement might be to its prime grade value. In this method, n measurements are used to classify each animal. Each grade has a nominal normalized measurement value for each of the n measurements. This results in a point for each grade in n-measurement space. The set of measurements for an unknown animal give it a point in the same measurement space. Using this method, the automated grading amounts to simply finding the grade point which lies closest to the unknown animal point- ¶0729-0735).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan with the teachings of Marshall accurate and convenient methods to evaluate fat content or lean:fat ratios (Marshall- ¶0167).
Regarding claim 10, Ivan teaches:
10. An information providing server that provides animal entity status information collected from at least one animal entity status information providing device(i.e. The present invention relates to a method and to a device for automatically calculating parameters of an object, for instance the size, dimensions, body part dimensions, body features, volume, weight and other related data allowing defining said object, which preferably would be an animal such as a livestock animal- Col 1, line 15-21), the information providing server (i.e. In accordance with yet another embodiment, the calculation of the size of a pixel of the object in the acquired and segmented two-dimensional image and the calculation of the several parameters is performed by a remote server including at least one processing unit. This is done by the remote server previously receiving at least the acquired two dimensional image, through a communication network such as internet, via a local area network (LAN) or a general wide area network (WAN), or any other communication means… In addition, the remote server can built a map to represent a spatial distribution of the calculated parameters, to monitor a temporal evolution of the calculated parameters, or to perform a combination of the represented spatial distribution and monitored temporal distribution. The built map then can be used to optimize at least pig growth, crop management, turf management, compost management, stock management, stock pricing, vision attention, marketing of products, mapping of pig production or reliability of producers, overall yield for a year and/or a predicted yield for the next season or years- col 3 line 54 to col 4 line 6) comprising:
a communication module (i.e. a communication network such as internet, via a local area network (LAN) or a general wide area network (WAN), or any other communication means- col 3, line 60-62); a memory configured to store an animal entity status information providing program; and a processor configured to execute the animal entity status information providing program (i.e. a remote server including at least one processing unit- col 3, line 54-62), wherein as the animal entity status information providing program is executed by the processor, the at least one animal entity status information providing program collects animal entity status information from at least one animal entity status information providing device,
the animal entity status information includes an estimated weight of the animal entities collected by the at least one animal entity status information providing device for a predetermined period of time, and the estimated weight is collected through an operation of calculating, by the animal entity status information providing device, sizes of the animal entities by using a scale set through camera calibration, and estimating, by the at least one animal entity status information providing device, weights of the animal entities based on the sizes of the animal entities (i.e. entities (i.e. the size of a pixel of the object in the acquired and segmented two dimensional image taking into account the distance between the object and the two-dimensional camera; and calculating, by a second means, several parameters of the object including at least the size, dimensions, body part dimensions, body features (including a body fat index), weight and/or volume by using said calculated size of the pixel and an a priori model of the object- col 3, line 27-34… Once the data has been processed (step 93) it is transmitted to the proposed device 100 allowing said computing software application displaying (step 94) the calculated value of the parameter- col 8, line 30-33).
the at least one animal entity status information providing device identifies an animal entity based on characteristic information of the animal entity extracted from image data or a color change of pixels in the image data(i.e. The method comprising: acquiring, by a two-dimensional camera (7), in a scene, a two dimensional image of at least one object (200); identifying the object (200) within the acquired two dimensional image; calculating, by a first means, the size of a pixel of the object (200) in the acquired and segmented two dimensional image taking into account the distance between the object (200) and the two-dimensional camera (7); and calculating, by a second means, several parameters including at least the size, dimensions, body part dimensions, body features, weight and/or volume of the object (200) by using said calculated size of the pixel and an a priori model of the object (200), wherein said a priori model includes information linking different parts, contours or shapes representative of several objects (200), previously acquired with a two-dimensional camera, with several parameters said several objects (200).- Abstract),
However, Ivan does not teach explicitly:
provides the at least one animal entity status information through a user interface of a user terminal in response to a request of an external user terminal, a number of animal entities; the number of animal entities is collected through an operation of counting the animal entities by identifying, by the at least one animal entity status information providing device, the animal entities in an image-capturing region of a camera; and counts the animal entity,
the camera is fixed on an upper portion of the straight passage.
In the same field of endeavor, Marshall teaches:
provides the at least one animal entity status information through a user interface of a user terminal in response to a request of an external user terminal (i.e. The graphical user interface is also suitable for other related search and browsing and selection functions. The applications program governs the display of the specific search criteria. Information can be previewed in the program window, a first sub window and a second sub window and through one or more source windows. The windows can contain a variety of information including graphically rendered objects, video media and externally acquired media from a media server. In one embodiment the graphical user interface can also be controlled from an applications program run on a remote server- ¶1368… The displays and interface pages, described herein, are meant to be representative of the various types of acquired data available to the users. It is understood that similar user interfaces may also be implemented using various toolbars, icons, related program applications, operating systems and external data processing servers while still falling within the scope of this invention- 1532), a number of animal entities; the number of animal entities is collected through an operation of counting the animal entities by identifying, by the at least one the animal entity status information providing device, the animal entities in an image-capturing region of a camera (i.e. This color, still image 520 of the target animal may be used for future identification of the animal and visual information such as color or breed. The input data region of this page also includes additional information about the target animal such as…b) an animal identification (I.D.) number code- ¶1389-1391… automatic entry from electromagnetic or thermal scanning of an animal I.D. number or code located somewhere on the target animal- ¶1405…The number of animals scanned is counted in the lower left of the page (704). The count is displayed as both a truck count and a cumulative count for the given owner- ¶1432… Since the automated data acquisition may be triggered by either analysis of a live video camera or the output of a proximity detector, the trigger technology is identified by mouse clicking over the appropriate box in region (972). Similarly, the trigger mode (973) and the I.D. method (974) may also be selected- ¶1462), and counts the animal entity(i.e. The method comprising: acquiring, by a two-dimensional camera (7), in a scene, a two dimensional image of at least one object (200); identifying the object (200) within the acquired two dimensional image; calculating, by a first means, the size of a pixel of the object (200) in the acquired and segmented two dimensional image taking into account the distance between the object (200) and the two-dimensional camera (7); and calculating, by a second means, several parameters including at least the size, dimensions, body part dimensions, body features, weight and/or volume of the object (200) by using said calculated size of the pixel and an a priori model of the object (200), wherein said a priori model includes information linking different parts, contours or shapes representative of several objects (200), previously acquired with a two-dimensional camera, with several parameters said several objects (200)- Abstract). ,
the camera is fixed on an upper portion of the straight passage (i.e. As indicated in FIG. 2-1, the data processing means of the invention for processing image data from the cameras 22, 24, 26, and 28- ¶0651).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan with the teachings of Marshall accurate and convenient methods to evaluate fat content or lean:fat ratios (Marshall- ¶0167).
However, Ivan and Marshall do not teach explicitly:
an image distortion is corrected based on an image obtained by capturing a checkerboard placed in a region of a straight passage through which an animal entity passes as the image- capturing region and set the scale, and calculates a scale for an actual length to a pixel by comparing a length of a unit line segment of the checkerboard with a number of pixels representing the unit line segment of the checkerboard in image data obtained by correcting image distortion.
In the same field of endeavor, Fausto teaches:
an image distortion is corrected based on an image obtained by capturing a checkerboard placed in a region of a straight passage through which an animal entity passes as the image- capturing region and set the scale, and calculates a scale for an actual length to a pixel by comparing a length of a unit line segment of the checkerboard with a number of pixels representing the unit line segment of the checkerboard in image data obtained by correcting image distortion (i.e. The calibration begins with the acquisition of the cube in a standard position, as shown in FIG. 12. The corners of the three checkerboards are then detected for providing camera 100 calibration data. The preferred camera 100 calibration method employs several fully automatic phases during which patterns of increasing complexity are detected. But, depending on the visual feedback, the user decides at which phase to begin and provides input clicks accordingly. The full process is applied independently to each checkerboard. Each phase begins with a known pattern defined by a certain number of points. In phase 1, the user clicks a triangle (a, b, c) representing a normal basis of the checkerboard. Then, from the extrapolated parallelogram (a, b, d, c), and using an automatic algorithm, each of the four corners is automatically adjusted so that the quadrangle matches "perfectly" the corresponding image edges. This is done by maximizing the integral of the magnitude of the image intensity gradient along the edges of the quadrangle pattern. The integral can be maximized by using a gradient ascent algorithm in multiple image scales to avoid irrelevant local minima. The initial parallelogram is allowed to deform as a general quadrangle because the camera 100 projection does not preserve parallelism. Each additional phase starts either using the known corners resulting from the previous phase or, if the user selects, by the same corners but specified manually. During the second phase the known quadrangle is extended by linear extrapolation to form the quadrangle (a, e, f, g). Using the same optimization algorithm, these four corners are then fitted automatically to the image data. The corner h is then initialized by quadratic extrapolation using a, b and e, accounting to some extent for projective depth distortion. The corners i and j are initialized similarly and the quadrangle (a, h, i, j) is automatically fitted. This process is repeated until all the corners (a, b, e, h, . . . , k), (a, d, f, i, . . . , l) and (a, c, g, j, . . . , m) are found. During the third phase, using linear, and whenever possible quadratic extrapolations, followed by automatic fitting to image data, eventually all the corners of the boundary of the checkerboard are detected. It is possible to do so by only considering a quadrangle with edges at least as large as half of the full checkerboard side. Avoiding shorter edges has been found to increase robustness. During the fourth phase, and starting from the known boundary corners, all the inside corners are initialized by simply intersecting "horizontal" and "vertical" lines. Due to camera 100 lens radial distortion, the "lines" are actually slightly curved, and deviations from the ideal line on the order of three pixels can be observed. This is accounted for during the fifth phase by fitting individual cross-hair patterns for each corner. The cross-hair pattern (s, t, u, v) is initialized and automatically fitted. The only allowed degrees of freedom during the optimization are the translation and the angle between su and tv. Furthermore, for each cross-hair pattern, this algorithm is repeated a number of times, starting from randomly selected nearby initial guesses. The corresponding corner is only considered as successfully detected if all of these tries yield the same result, up to some fraction e.g., one tenth, of a pixel. This is one suitable technique to enforce the significance of the detected corner. Finally, the detected corners are displayed, superimposed with the checkerboard image. The user then accepts or rejects the achieved detection, thereby completing the camera calibration procedure- ¶0033).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan and Marshall with the teachings of Fausto to achieve sub-millimeter accuracy so that when scans are registered and combined with the colors, the fine-detailed normals are not blurred (Fausto- ¶0032).
Regarding claim 11, Ivan, Marshall and Fausto teach all the limitations of claim 10.
However, Ivan does not teach explicitly:
wherein the animal entity status information providing program causes the camera, to be selected through a first interface block executed by the user terminal, causes a particular image captured by the camera selected through the first interface block to be output through a second interface block, and causes information on an operation state of the camera selected through the first interface block to be output through a third interface block.
In the same field of endeavor, Marshall teaches:
wherein the animal entity status information providing program causes the camera, to be selected through a first interface block executed by the user terminal, causes a particular image captured by the camera selected through the first interface block to be output through a second interface block, and causes information on an operation state of the camera selected through the first interface block to be output through a third interface block (i.e. Feedlot housing practices are very diverse from farm to farm, ranging from open dry yards, where protection is provided from inclement weather, to indoor confinement housing- ¶1305… c) the location of the data acquisition (i.e., farm, city, state)- ¶01392).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan with the teachings of Marshall accurate and convenient methods to evaluate fat content or lean:fat ratios (Marshall- ¶0167).
Regarding claim 15, method claim 15 corresponds to apparatus claim 1, and therefore is also rejected for the same reasons of obviousness as listed above.
Regarding claim 18, method claim 18 corresponds to apparatus claim 5, and therefore is also rejected for the same reasons of obviousness as listed above.
Claims 6, 7, 8, 14, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Ivan Amat Roldan et al. [US 10249054 B2: already of record] in view of Marshall S. Kriesel et al. [US 20050257748 A1: already of record] further in view of Fausto Bernardini et al. [US 20040217260 A1: already of record] and even further in view of Thomas Banhazi et al. [US 20160012278 A1: already of record].
Regarding claim 6, Ivan, Marshall and Fausto teach all the limitations of claim 1.
However, Ivan, Marshall and Fausto not teach explicitly:
wherein the animal entity status information generation program estimates the weights of the animal entities by inputting information on the calculated sizes of the animal entities to a lookup table representing a correlation between the sizes and weights of the animal entities.
In the same field of endeavor, Thomas teaches:
wherein the animal entity status information generation program estimates the weights of the animal entities by inputting information on the calculated sizes of the animal entities to a lookup table representing a correlation between the sizes and weights of the animal entities (i.e. The GUI includes an input interface to assist with valid weight prediction by setting the parameter values associated with expected animal dimensions. The GUI receives data relating to the sizes and distances described above with reference to the feature extraction process 900 as shown in FIGS. 32A and 32B, for example, “ML” corresponds to the maximum length described above with reference to FIG. 27. The parameters generated using data from the input GUI can be represented by a lookup table (LUT) which contains upper body measurements which are used to limit the shapes of potential contours detectable in the image to be within expected peak dimensions for the predicted weight: when an output weight is generated, this value is cross-referenced with the generated size information in the LUT and if the measurements are not within the selected parameter value limits, this frame is rejected. The limit on possible values as a relationship between total image area inside the contour and estimated weight can be shown on a graph with one line indicating a lower limit and a second line indicating an upper limit on acceptable values, as shown in FIGS. 32A and 32B- ¶0179).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan, Marshall and Fausto with the teachings of Thomas to improve productivity, efficiency, profitability, safety and animal health in farms (Thomas- ¶0092).
Regarding claim 7, Ivan, Marshall, Fausto and Thomas teach all the limitations of claim 6.
However, Ivan, Marshall and Fausto do not teach explicitly:
wherein the animal entity status information generation program stores characteristic information for each type of animal entities, compares the stored characteristic information for each type of animal entities with characteristic information on each individual entity identified from the image data, and determines a type of an animal of the identified animal entity, the lookup tables includes multiple lookup tables depending on types of animal, and the weights of the animal entities are estimated by inputting the information on the calculated sizes of the animal entities to the lookup table corresponding to the determined type of animal.
In the same field of endeavor, Thomas teaches:
wherein the animal entity status information generation program stores characteristic information for each type of animal entities (i.e. The animal 106 can be a farmed or livestock animal, for example a cow, a pig, a sheep, a fish, etc- ¶0094), compares the stored characteristic information for each type of animal entities with characteristic information on each individual entity identified from the image data, and determines a type of an animal of the identified animal entity, the lookup tables includes multiple lookup tables depending on types of animal, and the weights of the animal entities are estimated by inputting the information on the calculated sizes of the animal entities to the lookup table corresponding to the determined type of animal (i.e. The GUI includes an input interface to assist with valid weight prediction by setting the parameter values associated with expected animal dimensions. The GUI receives data relating to the sizes and distances described above with reference to the feature extraction process 900 as shown in FIGS. 32A and 32B, for example, “ML” corresponds to the maximum length described above with reference to FIG. 27. The parameters generated using data from the input GUI can be represented by a lookup table (LUT) which contains upper body measurements which are used to limit the shapes of potential contours detectable in the image to be within expected peak dimensions for the predicted weight: when an output weight is generated, this value is cross-referenced with the generated size information in the LUT and if the measurements are not within the selected parameter value limits, this frame is rejected. The limit on possible values as a relationship between total image area inside the contour and estimated weight can be shown on a graph with one line indicating a lower limit and a second line indicating an upper limit on acceptable values, as shown in FIGS. 32A and 32B- ¶0179).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan, Marshall and Fausto with the teachings of Thomas to improve productivity, efficiency, profitability, safety and animal health in farms (Thomas- ¶0092).
Regarding claim 8, Ivan, Marshall and Fausto teach all the limitations of claim 1.
However, Ivan, Marshall and Fausto do not teach explicitly:
wherein the animal entity status information generation program further outputs, as status information of the animal entities, an average of the estimated weights of the counted animal entities.
In the same field of endeavor, Thomas teaches:
wherein the animal entity status information generation program further outputs, as status information of the animal entities, an average of the estimated weights of the counted animal entities (i.e. Using the example process 3500, a trial was conducted with 11 finisher pigs. When animal weight was measured using both scales and the system 100, the error in group average weight determined by the example process 3500 was less than about 2 kg, and only one day had an error greater than 1 kg, as shown in FIG. 45 (the scales results are represented by isolated points 4502, and the results from the system 100 are represented by connected points 4504). Each measure of group average weight was the average weight of all samples acquired of the 11 animals across the course of a day- ¶0204).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan, Marshall and Fausto with the teachings of Thomas to improve productivity, efficiency, profitability, safety and animal health in farms (Thomas- ¶0092).
Regarding claim 14, Ivan, Marshall and Fausto teach all the limitations of claim 10.
However, Ivan does not teach explicitly:
displays the number and a distribution status of the animal entities corresponding to an under grade, a suitable grade, and an exceeding grade by considering a normal distribution.
In the same field of endeavor, Marshall teaches:
displays the number and a distribution status of the animal entities corresponding to an under grade, a suitable grade, and an exceeding grade by considering a normal distribution (i.e. One of the more useful analyses and displays for this type of data is a histogram as shown in FIG. 2-29. From such an analysis can be learned the distribution type (i.e., normal or bimodal), the mean, median, and standard deviation (normal distribution)- ¶0303-304, ¶1251).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan with the teachings of Marshall accurate and convenient methods to evaluate fat content or lean:fat ratios (Marshall- ¶0167).
However, Ivan, Marshall and Fausto do not teach explicitly:
wherein the animal entity status information providing program displays information on the number of animal entities, an average weight, or a standard deviation for measurements performed during a certain period in a farm selected by a user through a statistical information providing interface executed by the user terminal.
In the same field of endeavor, Thomas teaches:
wherein the animal entity status information providing program displays information on the number of animal entities, an average weight (i.e. Using the example process 3500, a trial was conducted with 11 finisher pigs. When animal weight was measured using both scales and the system 100, the error in group average weight determined by the example process 3500 was less than about 2 kg, and only one day had an error greater than 1 kg, as shown in FIG. 45 (the scales results are represented by isolated points 4502, and the results from the system 100 are represented by connected points 4504). Each measure of group average weight was the average weight of all samples acquired of the 11 animals across the course of a day- ¶0204), or a standard deviation for measurements performed during a certain period in a farm selected by a user through a statistical information providing interface executed by the user terminal.
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan, Marshall and Fausto with the teachings of Thomas to improve productivity, efficiency, profitability, safety and animal health in farms (Thomas- ¶0092).
Regarding claim 19, method claim 19 corresponds to apparatus claim 6, and therefore is also rejected for the same reasons of obviousness as listed above.
Regarding claim 20, method claim 20 corresponds to apparatus claim 7, and therefore is also rejected for the same reasons of obviousness as listed above.
Claims 12-13 are rejected under 35 U.S.C. 103 as being unpatentable over Ivan Amat Roldan et al. [US 10249054 B2: already of record] in view of Marshall S. Kriesel et al. [US 20050257748 A1: already of record] further in view of Fausto Bernardini et al. [US 20040217260 A1: already of record] and even further in view of Sung Bok Kwak et al. [US 20120326874 A1: already of record].
Regarding claim 12, Ivan, Marshall and Fausto teach all the limitations of claim 10.
However, Ivan, Marshall and Fausto do not teach explicitly:
wherein the animal entity status information providing program displays a number of tasks including corresponding animal entity status information for each day through a calendar-based interface executed by the user terminal, and displays a report for each task in a form of a list through a report output interface block adjacent to the calendar-based interface, the report for each task includes particular image data captured while a corresponding task is performed, and information on a number and the estimated weight of the animal entities counted during the corresponding task, and the report for each task is generated in first units of time previously determined by a user, or based on second units of time from a time when the animal entities begin to be identified to a time when identification ends.
In the same field of endeavor, Sung teaches:
wherein the animal entity status information providing program displays a number of tasks including corresponding animal entity status information for each day through a calendar-based interface executed by the user termina (i.e. One or more embodiments of the present invention may be particularly useful for alerting a farmer for a potentially-infectious disease in a particular farm animal attached with an RFID tag, based on its reduced activity levels to the activity measurement zone (AMZ) (e.g. 209). The early alert system and method may alert the farmer by a periodic communication method such as a periodic email report (i.e. hourly, daily, weekly, and etc.) or by dynamic event triggers. In dynamic event trigger instances, an e-mail, a telephone call, a text message, a display terminal alert, or any other desirable dynamically-triggered alert methods may be triggered by an alarming event, such as reaching an alert trigger point for a particular animal attached with an RFID tag- ¶0053)l, and displays a report for each task in a form of a list through a report output interface block adjacent to the calendar-based interface, the report for each task includes particular image data captured while a corresponding task is performed, and information on a number and the estimated weight of the animal entities counted during the corresponding task, and the report for each task is generated in first units of time previously determined by a user, or based on second units of time from a time when the animal entities begin to be identified to a time when identification ends (i.e. FIG. 15 shows an example of a report table format (1500) for an early alert system for livestock disease detection in accordance with an embodiment of the invention. In this embodiment of the invention, the report table format (1500) is generated, filtered, and/or refined by the networking and main controller system (e.g. 911 of FIG. 9), the RFID scanning system (e.g. 931 of FIG. 9), and/or the database and web interface system (e.g. 919 of FIG. 9). The report table format (1500) may contain a list of electronic product codes (EPC's) or unique RFID tag numbers (e.g. "B10001," "B10002," and etc.) in a first column (1501). The report table format (1500) may also contain a list of activity measurement zone (AMZ) accesses by a particular animal defined by a particular EPC or RFID tag number. Preferably, this list of AMZ accesses is further categorized by time, as shown in a second column (1503) in the report table format (1500). In a preferred embodiment of the invention, AMZ accesses for an animal attached with a specific RFID tag may be recorded periodically (e.g. every two minutes) and categorized in the second'column (1503) accordingly. In addition, the report table format (1500) may also contain a total count of AMZ accesses by a particular animal, as shown in a third column (1505). As previously described in other figures, a sudden drop in a total count of AMZ accesses by a particular animal may indicate a potential health problem for that animal. An embodiment of the early alert system for livestock disease detection as described in the present invention is able to flag a particularly alarming drop in AMZ accesses by a potentially sick animal, and report to a user of the early alert system in the report table format (1500) as shown in FIG. 15, or in another method of alert, such as an email alert, a text alert, or a telephone call alert- ¶0111).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan, Marshall and Fausto with the teachings of Sung to improve yield and efficiency of their livestock farms within available spaces (Sung- ¶0002).
Regarding claim 13, Ivan, Marshall and Fausto teach all the limitations of claim 12.
However, Ivan, Marshall and Fausto do not teach explicitly:
wherein the report output interface block is configured to output a highlight indicating a main grade for each report when the counted number of the animal entities is greater than a threshold.
In the same field of endeavor, Sung teaches:
wherein the report output interface block is configured to output a highlight indicating a main grade for each report when the counted number of the animal entities is greater than a threshold (i.e. The unique tag identification code (601) for the particular animal is also typically associated with other pieces of information, such as a type/grade of the animal (603), date of birth (605), gender (607), owner (609), and vaccine records (611) for the particular animal. In addition, the RFID tag, a computer server, and an analytical program module executed on the computer server may also keep records of castration because the completion of castration may impact a particular animal's behavior- ¶0070).
It would have been obvious to one with ordinary skill in the art before the effective filing date of the claimed invention, to modify the teachings of Ivan, Marshall and Fausto with the teachings of Sung to improve yield and efficiency of their livestock farms within available spaces (Sung- ¶0002).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to CLIFFORD HILAIRE whose telephone number is (571)272-8397. The examiner can normally be reached 5:30-1400.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, SATH V PERUNGAVOOR can be reached at (571)272-7455. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CLIFFORD HILAIRE
Primary Examiner
Art Unit 2488
/CLIFFORD HILAIRE/Primary Examiner, Art Unit 2488
Prosecution Insights