DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 17 and 33-34 are objected to because of the following informalities:
In claim 17, Examiner suggests correction of “wherein the predetermined events comprise strike events, abrupt movements, non-routing swimming events and routing swimming events” to read as “wherein the predetermined events comprise strike events, abrupt movements, non-routine swimming events and routine swimming events”, consistent with Applicant’s specification at ¶0050.
Each of claims 33-34 recites “The system of claim…” which should be corrected to read as “The method of claim…” as claim 33 is dependent from claim 32, which is dependent of base claim 18, which is a method claim.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 4, 13-16, 18, 21 and 30-33 are rejected under 35 U.S.C. 103 as being unpatentable over “Automated detection of feeding strikes by larval fish using continuous high-speed digital video: a novel method to extract quantitative data from fast, sparse kinematic events” (hereinafter “Shamur”, published 2016; applicant-submitted prior art) in view of U.S. PG PUB. 2020/0096434 A1 (hereinafter “Deran”; applicant-submitted prior art).
Regarding claim 1, a machine vision system for larval aquatic animal rearing, the system (Shamur, p. 1609, Experimental setup; “Visualization of feeding larvae was done using a continuous high-speed digital video system”) comprising:
a high-speed video camera comprising a lens, the lens exhibiting a high depth of field (DOF) and a microscopic resolution (Shamur, p.1608, Introduction, second paragraph, p.1609, right column, Experimental setup; “a continuous high-speed digital video system (Vieworks VC-4MC-M/C180), operating at 240 frames s−1 with a resolution of 2048×1024 pixels (Holzman et al., 2015). The camera was connected to a PC, and controlled by Streampix 5 video acquisition software (Norpix, Montréal, Canada). A 25 mm f/1.4 C-mount lens (Avenir CCTV lens, Seiko Optical, Mong Kok, Hong Kong) was mounted on an 8 mm extension tube, providing a field of view of 15×28×3 mm (height×width×depth) at f=5.6.” The camera system is configured to film and detect minute fish larvae while overcoming the problem identified in the Introduction of conventionally needing high magnification optics with a small depth-of-field and limited visualized area to film the minute larvae. Thus, the disclosed camera has a high depth of field and microscopic resolution, as claimed.);
a processor in communication with the high-speed video camera, wherein the high-speed video camera is arranged to continuously capture video of a predetermined volume and transmit the captured video to the processor (Shamur, p.1609, right column, Experimental setup; “The camera was connected to a PC, and controlled by Streampix 5 video acquisition software (Norpix, Montréal, Canada).”), and
wherein the processor is arranged to apply machine learning (Shamur, p.1610-1613, Fig. 2, Classification of feeding strikes, Stages A-D):
isolate individual aquatic animal within the video (Shamur, Figs. 2-3, p.1611, Stage A, video pre-processing and fish localization; “Our processing thus began by attempting to remove much of this clutter. We first applied a standard image segmentation technique (Otsu, 1975), which provides a binary separation of the video to foreground/background pixels, used to separate the background from noise and fish blobs”);
identify at least one predetermined activity parameter and/or at least one predetermined morphological anomaly of the isolated aquatic animal (Shamur, Fig. 2, Video representation by action descriptors, p. 1612-1613, Stage C; “The pose-normalized video clips produced in the previous step are next converted to robust representations (descriptors), whose function is to represent actions appearing in videos…Each descriptor is produced by an algorithm that represents (describes) a video clip based on features of the image sequence”); and
output the identified at least one predetermined activity parameter and/or the at least one predetermined morphological anomaly of the isolated aquatic animal (Shamur, Fig. 2, Classification, p. 1613, Stage D; “Binary classification of each clip, Vk, as representing either interaction or non-interaction with prey, was performed”).
Shamur does not expressly teach the limitations as further claimed, but, in an analogous field of endeavor, Deran does as follows.
Deran teaches:
wherein the processor is arranged to apply one or more neural networks to the captured video (Deran, ¶0110-0116; “The AI classifier 202 classifies full input images 207 from the camera, in one configuration outputting classifications for the entire image or in another configuration outputting a set of regions of interest 208, each containing classifications, or in another configuration, outputting both.” The AI classifier includes one or more neural networks.); and
a watertight housing, the high-speed video camera positioned within the watertight housing (Deran, ¶0325; “A remote system fits in a suitcase-size Pelican-type box, which is water-tight”).
Deran is considered analogous art because it pertains to classification of aquatic particles using a neural network. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system taught by Shamur to include neural networks for object/image classification and a watertight housing for the high-speed camera, as taught by Deran, in order to more accurately classify the fish larvae present in the image while protecting the camera setup from water damage.
Regarding claim 4, claim 1 is incorporated, and as established above, the combination of Shamur in view of Deran teaches the system of claim 1, specifically a high-speed camera configured to continuously capture video within a water tank with a high depth of field.
The combination of Shamur and Deran does not disclose expressly teach “wherein the high DOF allows to continuously capture video from at least 125 cm3 of water.”
Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to use the camera of Shamur in the combination to continuously capture video from at least 125 cm3 of water. Applicant has not disclosed that continuously capturing video from at least 125 cm3 of water provides an advantage, is used for a particular purpose, or solves a stated problem. One of ordinary skill in the art, furthermore, would have expected Applicant’s invention to perform equally well with the high speed camera taught by Shamur in the combination because the camera of Shamur achieves the same desired outcome of acquiring video images with a high depth of field, and therefore would capture the recited video with a reasonable expectation of success.
Therefore, it would have been obvious to one of ordinary skill in this art to modify the camera in the combination of Shamur in view of Deran to continuously capture video from at least 125 cm3 of water to obtain the invention as specified in claim 4.
Regarding claim 13, claim 1 is incorporated, and Shamur in the combination further teaches wherein the at least one predetermined activity parameter is selected from the group consisting of: cohort size; activity level; feeding performance; and food preference (Shamur, p.1616, last paragraph; “Our method can be employed in a wide range of studies on larval feeding: the effect of inter- and intra-specific competition, food preferences and feeding selectivity, prey escape response and predator–prey co-evolution.”).
Regarding claim 14, claim 1 is incorporated, and since Examiner has selected the “identify at least one predetermined activity parameter” condition in Claim 1 from the limitation ““identify at least one predetermined activity parameter and/or at least one predetermined morphological anomaly” and presented prior art references that teach the “identify at least one predetermined activity parameter” condition, this claim which represents an alternative condition does not need to be analyzed.
Regarding claim 15, claim 1 is incorporated, and Shamur in the combination further teaches wherein the processor is configured to apply an action classifier to the captured video to classify predetermined portions of the captured video into different predetermined events, the at least one predetermined activity comprising the different predetermined events, wherein the action classifier is trained by the one or more neural networks (Shamur, Fig. 2, Classification, p. 1613, Stage D; “Binary classification of each clip, Vk, as representing either interaction or non-interaction with prey, was performed… multiple learners (of the four descriptors mentioned in our case: MIP, STIP, VIF and MBH) are combined to solve the same problem (classification as feeding or non-feeding).”).
Regarding claim 16, claim 15 is incorporated, and Shamur in the combination further teaches wherein the predetermined events comprise swim events and strike events (Shamur, Fig. 2, Classification, p. 1613, Stage D; “Binary classification of each clip, Vk, as representing either interaction or non-interaction with prey, was performed…” Interaction with prey is read here as a “strike event”, and non-interaction is read as the claimed “swim event”).
Claim 18 recites a method having features which correspond to the elements recited in system claim 1, the rejection of which is applicable here.
Claim 21 recites a method having features which correspond to the elements recited in system claim 4, the rejection of which is applicable here.
Claim 30 recites a method having features which correspond to the elements recited in system claim 13, the rejection of which is applicable here.
Claim 31 recites a method having features which correspond to the elements recited in system claim 14, the rejection of which is applicable here.
Claim 32 recites a method having features which correspond to the elements recited in system claim 15, the rejection of which is applicable here.
Claim 33 recites a method having features which correspond to the elements recited in system claim 16, the rejection of which is applicable here.
Claims 8 and 25 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Shamur in view of Deran, as applied to claims 1 and 18 above, and further in view of U.S. PG PUB. 2016/0183502 A1 (hereinafter “Tanase”).
Regarding claim 8, claim 1 is incorporated, and Shamur in the combination further teaches a light emitting diode (LED) backlit illumination panel, the illumination panel positioned between the housing and the predetermined volume (Shamur, p.1609, right column, Experimental setup “We used backlit illumination, using an array of 16 white LEDs”).
The combination of Shamur in view of Deran does not expressly teach the limitations as further claimed, but, in an analogous field of endeavor, Tanase does as follows.
Tanase teaches wherein the illumination panel is submersible (Tanase, ¶0021, 0037-0039; “the illumination system comprises a plurality of light sources, preferably (sets of) light emitting diodes, submersible or submersed in the water”).
Tanase is considered analogous art because it pertains to enhancing growth of aquatic animals. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system taught by the combination of Shamur and Deran to submerge the illumination panel, as taught by Tanase, in order to cause less stress for the fish during feeding (Tanase, ¶0039).
Claim 25 recites a method having features which correspond to the elements recited in system claim 8, the rejection of which is applicable here.
Claims 6, 10, 23 and 27 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Shamur in view of Deran, as applied to claims 1 and 18 above, and further in view of “3D-ZeF: A 3D Zebrafish Tracking Benchmark Dataset” (hereinafter “Pederson”; published 2020, applicant-submitted prior art).
Regarding claim 6, claim 1 is incorporated, and the combination of Shamur and Deran does not expressly teach the limitations as further claimed, but, in an analogous field of endeavor, Pederson does as follows.
Pederson teaches wherein the housing is positioned about 5 cm away from the predetermined volume (Pederson, Fig. 2, p.2424-2425, Sections 2—3.1; “The top and front cameras are GoPro Hero 5 and GoPro Hero 7, respectively. All the videos are recorded with a resolution of 2704 × 1520, 60 FPS, 1/60 s shutter speed, 400 ISO, and a linear field of view. However, the fish tank does not take up the entire image, therefore, the effective region of interest is approximately 1200 × 1200 and 1800 × 900 for the top- and front view, respectively.”).
Pederson is considered analogous art because it pertains to imaging aquatic animals for activity tracking. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system taught by the combination of Shamur and Deran to place the camera housing a certain distance away from the water tank, as taught by Pederson, in order to reduce reflections from the water surface (Pederson, Section 3.1).
Regarding claim 10, claim 1 is incorporated, and Shamur in the combination further teaches a diffuser (Shamur, p.1609, right column, Experimental setup “We used backlit illumination…with a white plastic diffuser.”).
The combination of Shamur in view of Deran does not expressly teach the limitations as further claimed, but, in an analogous field of endeavor, Pederson does as follows.
Pederson teaches a semi-transparent diffuser, the diffuser positioned such that the housing and the diffuser are on opposing sides of the predetermined volume (Pederson, Fig. 2, Experimental setup, p.2425, Section 3.1; “Semi-transparent plastic was attached to three out of four of the window panes in order to reduce reflections. Furthermore, the front camera was placed orthogonally to the water level, which reduced reflections from the water surface.”).
Pederson is considered analogous art because it pertains to imaging aquatic animals for activity tracking. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system taught by the combination of Shamur and Deran to include a semi-transparent plastic around the tank, as taught by Pederson, in order to reduce reflections from the water surface (Pederson, Section 3.1).
Claim 23 recites a method having features which correspond to the elements recited in system claim 6, the rejection of which is applicable here.
Claim 27 recites a method having features which correspond to the elements recited in system claim 10, the rejection of which is applicable here.
Claims 17 and 34 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Shamur in view of Deran, as applied to claims 1 and 18 above, and further in view of “DanioSense: Automated High-Throughput Quantification of Zebrafish Larvae Group Movement” (hereinafter “Wang”; published 2021).
Regarding claim 17, claim 16 is incorporated, and the combination of Shamur in view of Deran does not expressly teach the limitations as further claimed, but, in an analogous field of endeavor, Wang does as follows.
Wang teaches wherein the predetermined events comprise strike events, abrupt movements, non-routing swimming events and routing swimming events (Wang, Section VI.C., p.1066-1067; “Larval behavior patterns in the experiment were simply categorized into three types, namely, static, cruise, and burst”).
Wang is considered analogous art because it pertains to tracking and classification of fish larval movements. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the system taught by the combination of Shamur and Deran to include classifying the actions as the different behavior patterns, as taught by Wang, in order to generate more detailed quantification of larval morphology and movement (Wang, Introduction, last paragraph).
Claim 34 recites a method having features which correspond to the elements recited in system claim 17, the rejection of which is applicable here.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAMAH A BEG whose telephone number is (571)270-7912. The examiner can normally be reached M-F 9 AM - 5 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HENOK SHIFERAW can be reached at 571-272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAMAH A BEG/Primary Examiner, Art Unit 2676
Prosecution Insights