DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant’s arguments, filed 02/05/2026, with respect to 101 rejection have been fully considered and are persuasive. The 101 rejection of claims 1-19 has been withdrawn.
Applicant's arguments filed 02/05/2026 have been fully considered but they are not persuasive. In response to applicant arguments on page 10, the applicant argues subject matter that is not in the current claim amendment, “the present system actual changes over time, regardless of the color spectrum or grayscale value, whereby only meaningful temporal signals are amplified. Only meaningful temporal signals are amplified. The system analyzes the time-series data of each pixel and only amplifies true frame-to-frame variations, whether due to movement or color change. By identifying pixels that change from frame to frame, freezing all other pixels, and monitoring the pixel over a set period, improved classification and detection is possible.”. The amended subject matter added is broader in scope than argued. Prior art Savastinuk’433 shows in [0020] a digital still camera, configured to capture subsequent frames in rapid succession, or a video camera able to capture streaming video. Paragraph [0060] shows video camera captures information at a rate of at least 15-30 frames per second, as is typical for even low quality conventional video cameras, the number of data points will be more than adequate to determine variations due to heart rate. Paragraphs [0062]-[0063] amplification techniques involve isolating sections of video data (sequence of frames) (such as pixels) where changes are detected (for example, changes in color of an object or in movement of the object). Therefore the newly amendment subject matter for independent claim is still rejected by the previous prior art. See rejection below.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-9 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Savastinuk et al. (US 2015/0378433 A1) in view of Amat Roldan (US 2022/0254183 A1).
With respect to Claim 1, Savastinuk’433 shows a computer method for measuring heart rate in livestock (Paragraph [0096] to measure physiological sensor data of an individual animal of livestock shown in a herd on a camera feed), comprising:
receiving a video stream including one or more livestock (Paragraph [0096] individual animal of livestock shown in a herd on a camera feed);
determining, from the video stream, a digital identity of a livestock individual (Paragraph [0096] tracking such individuals, for example matching a particular individual animal of livestock shown in a herd on a camera feed);
finding, in frames of the video stream (Paragraph [0060] video camera captures information at a rate of at least 15-30 frames per second), a feature of the livestock individual (Paragraph [0096] a particular individual animal of livestock shown in a herd on a camera feed with physiological sensor data showing an irregular heartbeat) that exhibits heart rate-correlated periodic flushing (Figure 1A cameras 106 and paragraph [0021] The device may determine data corresponding to a physiological condition of the detected individuals using data from the device's cameras 106 (152). That data may include, for example, video data that shows the blood flow to the respective faces to detect their respective pulses. Paragraph [0052] As used herein a physiological condition is a condition or state of the body or bodily functions. Examples of physiological conditions include heart rate/pulse, sinus rhythm, breathing/respiratory rate, pupil dilation, blood flow to certain parts of the body, temperature, etc. The examples below illustrate user matching based on blood flow to the face, but other physiological conditions may also be used. In certain embodiments, the physiological condition may be one that is detectable by cameras of a device (such as detecting blood flow through color changes, breathing rate through expansion of a chest and/or nostrils, etc.). Paragraph [0060] The frequency at which the red colors change can be used to determine the use's approximate heartbeat, and the frequency must be within a determined range of potential heart beats in order for the user to be authenticated as an actual person. To filter out color changes that may not correspond to a pulse, a filter such as a band pass filter may be used to isolate color changes that have a frequency that generally corresponds to the possible range of frequencies of a human pulse.);
receiving a sequence of digital photographic frames at the feature (paragraph [0020] capture subsequent frames in rapid succession, or a video camera able to capture streaming video, paragraph [0060] video camera captures information at a rate of at least 15-30 frames per second, as is typical for even low quality conventional video cameras, the number of data points will be more than adequate to determine variations due to heart rate);
performing pixel amplification at the feature over a temporal sequence of frames to amplify the periodic flushing (Paragraphs [0058]-[0059] and [0062]-[0063] describes isolating particular pixels of video data for color amplification to ease detection of a physiological condition in frames of a video in sequence as shown in paragraph [0060]);
calculating a heart rate of the livestock individual from a sequence of the amplified periodic flushing (Paragraphs [0058]-[0059] and [0062]-[0063] describes isolating particular pixels of video data for color amplification to ease detection of a physiological condition describes as heart rate/beat); and
outputting information about the heart rate of the livestock individual [ ] (Paragraph [0096] describes to determine heart rate/beat of livestock for selecting the individual animal for veterinary care if needed).
Savastinuk’433 does not specifically disclose outputting information about the heart rate of the livestock individual on a physical device.
Amat Roldan’189 disclose outputting information about the heart rate of the livestock individual on a physical device (Figures 6-7 and paragraph [0042] describes displaying information on a physical device 118/119 via augmenting the information onto the real image of the livestock).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Savastinuk’433 to include outputting information about the heart rate of the livestock individual on a physical device method taught by Amat Roldan’189. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to allowing a technician/operator to identify the particular livestock (paragraph [0043]).
With regards to Claim 2, the combination of Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein receiving a video stream including one or more livestock in a pen includes tracking livestock as they mill about (in Amat Roldan: Figure 4A paragraphs [0049] and [0041] describes fixed mounted camera devices 110 per pen 105).
With regards to Claim 3, the combination of Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, further comprising: displaying a labeled image of the livestock on an electronic display (in Amat Roldan: Figures 6-7 and paragraph [0042] describes displaying information on a physical device 118/119 via augmenting (labeling) the information onto the real image of the livestock).
With regards to Claim 5, the combination of Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein outputting information about the heart rate of the livestock individual includes physically outputting information corresponding to the heart rate of the livestock individual (in Amat Roldan’183: Figures 6-7 and paragraph [0042] describes displaying information on a physical device 118/119 via augmenting the information onto the real image of the livestock).
With regards to Claim 6, the combination of Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein outputting information about the heart rate of the livestock individual includes displaying, on an electronic display (in Amat Roldan’183: Figures 6-7 and paragraph [0042] describes displaying information on a physical device 118/119 via augmenting the information onto the real image of the livestock), a notice that the individual livestock has a heart rate that meets a threshold heart rate indicative of physical distress (in Savastinuk’433: Paragraph [0096] matching a particular individual animal of livestock shown in a herd on a camera feed with physiological sensor data showing an irregular heartbeat may be useful for selecting the individual animal for veterinary care).
With regards to Claim 7, the combination of Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein outputting information about the heart rate of the livestock individual on an electronic display includes displaying an image (in Amat Roldan’183: Figures 6-7 and paragraph [0042] describes displaying information on a physical device 118/119 via augmenting the information onto the real image of the livestock), on an electronic display, of the individual livestock exhibiting the heart rate that meets a threshold heart rate indicative of physical distress (Savastinuk’433: Paragraph [0096] matching a particular individual animal of livestock shown in a herd on a camera feed with physiological sensor data showing an irregular heartbeat may be useful for selecting the individual animal for veterinary care).
With regards to Claim 8, combination of Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein outputting information about the heart rate of the livestock individual includes displaying, on an electronic display, an image of the one or more livestock as they mill about with the imaged of the individual livestock (in Amat Roldan’183: Figures 6-7 and paragraph [0042] describes displaying information on a physical device 118/119 via augmenting the information onto the real image of the livestock) exhibiting a heart rate highlighted (Savastinuk’433 Paragraph [0096] matching a particular individual animal of livestock shown in a herd on a camera feed with physiological sensor data showing an irregular heartbeat may be useful for selecting the individual animal for veterinary care).
With regards to Claim 9, the combination of Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein outputting information about the heart rate of the livestock individual on an electronic display includes outputting a message on a personal electronic device (in Amat Roldan’183: Figures 6-7 depict messages such as sick or “ok” displayed augmented onto the real image of the livestock).
With regards to Claim 12, the combination of Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein finding the feature that periodically flushes with heartbeat includes finding a snout of the livestock individual (in Savastinuk’433: paragraph [0052] the physiological condition may be one that is detectable by cameras of a device (such as detecting blood flow through color changes, breathing rate through expansion of a chest and/or nostrils, etc.)).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Savastinuk et al. (US 2015/0378433 A1) in view of Amat Roldan (US 2022/0254183 A1) further in view of Ramer et al. (US 2018/0160510 A1).
With regards to Claim 4, the combination of Savastinuk’433 and Amat Roldan’189 does not specifically show the computer method for measuring heart rate in livestock of claim 1, wherein receiving a video stream including the one or more livestock in the pen includes receiving a video stream from a hyperspectral digital image capture device.
Ramer’510 shows the computer method for measuring heart rate in livestock of claim 1, wherein receiving a video stream including the one or more livestock in the pen includes receiving a video stream from a hyperspectral digital image capture device (Figure 7 paragraph [0015]).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Savastinuk’433 and Amat Roldan’189 to include wherein receiving a video stream including the one or more livestock in the pen includes receiving a video stream from a hyperspectral digital image capture device method taught by Ramer’510. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to monitoring livestock (paragraph [0015]).
Claims 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Savastinuk et al. (US 2015/0378433 A1) in view of Amat Roldan (US 2022/0254183 A1) further in view of Hanks et al. (2018/0333244 A1).
With regards to Claim 10, the combination of Savastinuk’433 and Amat Roldan’189 does not specifically shows the computer method for measuring heart rate in livestock of claim 1, further comprising: storing the calculated heart rate in a computer readable non-transitory memory including previously calculated heart rates of the livestock individual.
Hanks’244 shows further comprising: storing the calculated heart rate in a computer readable non-transitory memory including previously calculated heart rates of the livestock individual (paragraph [0275] store heart rate 2004 as histories including three-to five days worth of historical data stored in computer readable media 216 figure 2).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Savastinuk’433 and Amat Roldan’189 to include storing the calculated heart rate in a computer readable non-transitory memory including previously calculated heart rates of the livestock individual method taught by Hanks’244. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to reduce the amount of computing resources and infrastructure required to transfer the animals because the animal's data can travel with the animal as the histories stored in the computer readable media (paragraph [0276]).
With regards to Claim 11, Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein comparing the calculated heart rate of the livestock individual to a heart rate threshold includes: [ ] ; and determining if the calculated heart rate is different than [ ] threshold calculated heart rates (in Savastinuk’433: Paragraph [0096] a particular individual animal of livestock showing an irregular (threshold) heartbeat); and wherein physically outputting the information corresponding to the heart rate of the first livestock individual (in Amat Roldan’189: Figures 6-7 and paragraph [0042] describes displaying information on a physical device 118/119 via augmenting the information onto the real image of the livestock).
Savastinuk’433 and Amat Roldan’189 does not specifically show comparing the calculated heart rate of the livestock individual to one or more previously calculated heart rates of the livestock individual and determining if the calculated heart rate is different than the previously calculated heart rates.
Hanks’244 shows comparing the calculated heart rate of the livestock individual to one or more previously calculated heart rates of the livestock individual (paragraph [0396] describes to compare the heart rate of the individual animal to multiple previous heart rates of the individual animal) and determining if the calculated heart rate is different than the previously calculated heart rates (Paragraph [0396] describes to determine, based on the above described comparison, whether the heart rate is considered stable, figure 24 2430)
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Savastinuk’433 and Amat Roldan’189 to include comparing the calculated heart rate of the livestock individual to one or more previously calculated heart rates of the livestock individual and determining if the calculated heart rate is different than the previously calculated heart rates method taught by Hanks’244. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to determining a stable heart rate (paragraph [0396]).
Claims 13-14 are rejected under 35 U.S.C. 103 as being unpatentable over Savastinuk et al. (US 2015/0378433 A1) in view of Amat Roldan (US 2022/0254183 A1) further in view of Jia et al. (US 2022/0164967).
With regards to Claim 13, Savastinuk’433 and Amat Roldan’189 does not specifically shows the computer method for measuring heart rate in livestock of claim 1, wherein performing pixel amplification at the feature to amplify the periodic flushing is performed using a Laplacian Pyramid.
Jia’967 shows wherein performing pixel amplification at the feature to amplify the periodic flushing is performed using a Laplacian Pyramid (Figures 6A-7C paragraphs [0131]-[0133] describes up-sampling processing using Laplacian pyramid with black and white/bright (negative) images of resolutions decreased gradually from bottom to top to obtain essential characteristics (of blood) of an image easier than compared to single resolution images).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Savastinuk’433 and Amat Roldan’189 to include performing pixel amplification at the feature to amplify the periodic flushing is performed using a Laplacian Pyramid method taught by Jia’967. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to obtaining essential characteristics easier as compared to single resolution images (paragraph [0131]).
With regards to Claim 14, Savastinuk’433, Amat Roldan’189 and Jia’967 shows the computer method for measuring heart rate in livestock of claim 13, wherein using the Laplacian Pyramid includes: creating negatives of each image at various resolutions; and performing image addition with the original image on a frame-to-frame basis (in Jia’967: Figures 6A-7C paragraphs [0131]-[0133] describes up-sampling processing using Laplacian pyramid with black and white/bright (negative) images of resolutions decreased gradually from bottom to top to obtain essential characteristics (of blood) of an image easier than compared to single resolution images).
Claims 15 is rejected under 35 U.S.C. 103 as being unpatentable over Savastinuk et al. (US 2015/0378433 A1) in view of Amat Roldan (US 2022/0254183 A1) further in view of Yoshizawa et al. (US 2018/0042486).
With regards to Claim 15, Savastinuk’433 and Amat Roldan’189 does not specifically shows the computer method for measuring heart rate in livestock of claim 1, wherein performing pixel amplification at the feature to magnify periodic flushing further comprises: removing high frequency noise from an image addition frame sequence using a maximally flat magnitude filter within a passband corresponding to a livestock heart rate range.
Yoshizawa’486 shows performing pixel amplification at the feature to magnify periodic flushing further comprises: removing high frequency noise from an image addition frame sequence using a maximally flat magnitude filter within a passband corresponding to a livestock heart rate range (Figure 8 paragraph [0147] applying a butterworth filter that passes a band of frequencies from .7-2Hz (maximally flat magnitude) corresponding to the heart rate range. The band pass removing frequencies above 2Hz (removing higher frequencies) attributed to body motions and respiratory fluctuations (noise)).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Savastinuk’433 and Amat Roldan’189 to include performing pixel amplification at the feature to magnify periodic flushing further comprises: removing high frequency noise from an image addition frame sequence using a maximally flat magnitude filter within a passband corresponding to a livestock heart rate range method taught by Yoshizawa’486. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to be able to allow passing only the heartbeat frequency as desired (paragraph [0147]).
Claim 17 is rejected under 35 U.S.C. 103 as being unpatentable over Savastinuk et al. (US 2015/0378433 A1) in view of Amat Roldan (US 2022/0254183 A1) further in view of Darrel et al. (US 8704668 B1).
With regards to Claim 17, Savastinuk’433 and Amat Roldan’189 shows the computer method for measuring heart rate in livestock of claim 1, wherein calculating the heart rate includes performing image analysis on one or more pixel-amplified video frame sequences [ ] (in Savastinuk’433: Paragraphs [0058]-[0059] and [0062]-[0063] describes isolating particular pixels of video data for color amplification to ease detection of a physiological condition).
Savastinuk’433 and Amat Roldan’189 does not specifically show the video frame sequences of a livestock from at least two video clips, wherein a first video clip captured at a wide angle and a second video clip is captured at a narrow angle.
Darrel’668 shows video frame sequences of a livestock from at least two video clips, wherein a first video clip captured at a wide angle and a second video clip is captured at a narrow angle (column 7, lines 8-21 the active cameras shall track a particular animal or animal type when detected in the environment. Active cameras may also be directed to focus on locations where particular animals or events have been detected from other sensor nodes. In a yet further embodiment, one set of wide angle or wide coverage sensors provides coarse detection of animals and activities, and another set of sensors for focused or narrow angle views provides detailed observation of animals and activities).
At the time of the invention, it would have been obvious to one of ordinary skill in the art before the effective filling date of the claim invention to modify Savastinuk’433 and Amat Roldan’189 to include video frame sequences of a livestock from at least two video clips, wherein a first video clip captured at a wide angle and a second video clip is captured at a narrow angle method taught by Darrel’668. The suggestion/motivation for doing so would have been to improve the system’s ability to be able to improve the different views of a desired animal or focus from the video using different angle views (in column 7, lines 8-21).
Allowable Subject Matter
Claims 18-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al. (US 2024/0193958 A1): paragraphs [0029] and [0037] a first field-of-view FOV1 of the first camera 110 and a second field-of-view FOV2 of the second camera 120 are partially overlapped. The second field-of-view FOV2 of the second camera 120 and a third imaging range FOV3 of the third camera 130 are partially overlapped. The third imaging range FOV3 of the third camera 130 and a fourth imaging range FOV4 of the fourth camera 140 are partially overlapped. The fourth imaging range FOV4 of the fourth camera 140 and the first field-of-view FOV1 of the first camera 110 are partially overlapped, wherein the object of interest can be an animal.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to IRIANA CRUZ whose telephone number is (571)270-3246. The examiner can normally be reached 10-6.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi M. Sarpong can be reached at (571) 270-3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/IRIANA CRUZ/Primary Examiner, Art Unit 2681
Prosecution Insights