DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-12 and 14-20 are currently pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6, and 8-11, 14, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (CN 106022459 A), hereafter referred to as “Chen” in view of Shiraishi (JP 2011047801 A), hereafter referred to as “Shiraishi.”
Regarding claim 1, Chen discloses a fish counting system (figs. 1-7) comprising:
an image acquisition unit (2, 3) configured to acquire a plurality of images by imaging over time an imaging area (6, 7, 8-1, 8-2, 9, and region of 1, fig. 1, which is captured by 2) in which fluid including a fish flows (figs. 1, 4-7, in which the system exists underwater);
an extraction unit configured to extract a fish for each image (machine translation, page 6, S1-S14); and
a counting unit configured to count a number of fish (machine translation, page 7, S2-S22); and
wherein:
the imaging area includes a first area (8-1, fig. 2) and a second area (8-2, fig. 2), and
the counting unit is configured to count the number of fish when the fish in the first area move to the second area (machine translation, page 7, S22), but does not explicitly teach:
a reporting unit configured to determine that an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal based on a variance of a flatness of the at least a single image exceeding a first threshold or a variance of a turbidity of the fluid in the at least a single image exceeding a second threshold, wherein: the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image, wherein: the variance of the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image.
Shiraishi teaches an imaging system (figs. 1-11 and attached machine translation, paragraph [0007]) including a reporting unit (machine translation, paragraphs [0089]-[0102]) configured to determine that an imaging condition is abnormal based on at least a single image (machine translation, paragraphs [0089]-[0102], see specifically, e.g., paragraph [0100]), the imaging condition is abnormal based on a variance of a turbidity of the fluid in the at least a single image exceeding a threshold (machine translation, paragraphs [0089]-[0102]), wherein:
the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image exceeding a second threshold (machine translation, paragraphs [0089]-[0102], see specifically, e.g., paragraphs [0093]-[0099]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen to include a reporting unit configured to determine that an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal based on a variance of a turbidity of the fluid in the at least a single image exceeding a second threshold, wherein: the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image, as taught by Shiraishi, in order to promptly notify a user if imaging conditions are inadequate to improve the quality and efficiency of the counting system (machine translation, paragraphs [0089] and [0100]).
Shiraishi further teaches calculating a variance of a parameter (machine translation, paragraph [0082]-[0083]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen in view of Shiraishi, such that the determination of an abnormal imaging condition is based on a variance of the turbidity, in order to monitor how the turbidity of the fluid fluctuates over time or across the imaged region to detection of subtle abnormalities in the fluid (machine translation, paragraph [0082]-[0083]).
The combined teachings of Chen and Shiraishi would result in the variance of the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image exceeding a second threshold.
Regarding claim 2, Chen as modified by Shiraishi teaches the fish counting system according to claim 1, and Chen further teaches:
a labeling unit configured to assign a label to an extracted fish to identify the fish (machine translation, page 7, S21-S22); and
a status setting unit configured to set a status to the fish to which the label is assigned (machine translation, page 8, paragraph beginning with “Figure 4-7”),
wherein:
the status setting unit is configured to set a status indicating a target of addition to the fish in the first area (machine translation, page 8, paragraph beginning with “Figure 4-7”), and
the counting unit is configured to count the fish when it is determined that the fish in the first area set as the target of addition moved to the second area (machine translation, page 8, paragraph beginning with “Figure 4-7”).
Regarding claim 3, Chen as modified by Shiraishi teaches the fish counting system according to claim 2, and Chen further teaches wherein the counting unit is configured to calculate the fish that moved through the second area to the first area when the fish is set as a target of subtraction (machine translation, page 8, paragraph beginning with “Figure 4-7”).
Regarding claim 4, Chen as modified by Shiraishi teaches the fish counting system according to claim 2, and Chen further teaches wherein the imaging area (6, 7, 8-1, 8-2, 9, and region of 1, fig. 1, which is captured by 2) has a third area (9, fig. 2) between the first (8-1) and second areas (8-2).
Regarding claim 5, Chen as modified by Shiraishi teaches the fish counting system according to claim 4, and Chen further teaches wherein:
when a first fish in an image of interest and a second fish in a past image one or more images before the image of interest satisfy a first tracking enabling condition (figs. 4-7 and machine translation, page 8, paragraph beginning with “Figure 4-7”), the labeling unit is configured to perform a tracking process to assign the same label as to the second fish to the first fish (machine translation, page 8, paragraph beginning with “Figure 4-7”), and
when the fish set as the target of addition moves to the second area after consecutively satisfying the first tracking enabling condition a predetermined number of times in the third area in the plurality of images, the counting unit is configured to count the fish (machine translation, page 8, paragraph beginning with “Figure 4-7”).
Regarding claim 6, Chen as modified by Shiraishi teaches the fish counting system according to claim 4, and Chen further teaches wherein:
when a first fish in an image of interest and a second fish in a past image one or more images before the image of interest satisfy a first tracking enabling condition, the labeling unit is configured to perform a tracking process to assign the same label as to the second fish to the first fish (figs. 4-7 and machine translation, page 8, paragraph beginning with “Figure 4-7”), and
the tracking process is performed on the fish set as the target of addition before being performed on the fish that is not set as the target of addition (machine translation, page 8, paragraph beginning with “Figure 4-7”).
Regarding claim 8, Chen as modified by Shiraishi teaches the fish counting system according to claim 5, and Chen further teaches wherein:
the labeling unit (machine translation, page 7, S21-S22) is configured to:
perform, when the first fish in an image of interest and the second fish in a past image one or more images before the image of interest satisfy the first tracking enabling condition, a tracking process to assign the same label as to the second fish to the first fish (figs. 4-7 and machine translation, page 8, paragraph beginning with “Figure 4-7”), and
assign a label as a new individual to a fifth fish to which a label was not assigned in the image of interest which was subjected to the tracking process (machine translation, page 6, S1-S14, and page 8, paragraph beginning with “Figure 4-7” disclosing that the system tracks multiple fish and their individual movements, which would require assigning unique labels),
the status setting unit is configured to set the same status as a fish closest to the fifth fish in a past image one image before the image of interest to the fifth fish (machine translation, pages 6-7, S14).
Regarding claim 9, Chen as modified by Shiraishi teaches the fish counting system according to claim 1, and Shiraishi further teaches wherein the reporting unit is configured to report to the outside based on a determination that the imaging condition is abnormal based on at least the single image that the imaging condition is abnormal (machine translation, paragraphs [0089]-[0102]).
Regarding claim 10, Chen teaches a fish counting method (machine translation, paragraphs 1-5 under “Summary of Invention”) comprising:
acquiring a plurality of images by imaging over time an imaging area (6, 7, 8-1, 8-2, 9, and region of 1, which is captured by 2, see figs. 1-2) in which fluid including a fish flows (figs. 1, 4-7, in which the system exists underwater),
extracting a fish for each image (machine translation, page 6, S1-S14),
counting a number of fish (machine translation, page 7, S2-S22),
wherein:
the imaging area includes a first area (8-1, fig. 2) and a second area (8-2, fig. 2), and the counting of the number of fish includes counting a fish when the fish in the first area moves to a second area (machine translation, page 7, S22),
but does not explicitly teach:
determining that an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal based on a variance of a flatness of the at least a single image exceeding a first threshold or a variance of a turbidity of the fluid in the at least a single image exceeding a second threshold, and
wherein:
the variance of a turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image.
Shiraishi teaches a fish imaging method (figs. 1-11 and attached machine translation, paragraphs [0007]-[0008]) including determining that an imaging condition is abnormal based on at least a single image (machine translation, paragraphs [0089]-[0102], see specifically, e.g., paragraph [0100]), the imaging condition is abnormal based on a turbidity of the fluid in the at least a single image exceeding a threshold (machine translation, paragraphs [0089]-[0102]), and
wherein:
the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image (machine translation, paragraphs [0089]-[0102], see specifically, e.g., paragraphs [0093]-[0099]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Chen to include the step of determining that an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal based on a turbidity of the fluid in the at least a single image exceeding a second threshold, and wherein: the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image, as taught by Shiraishi, in order to promptly notify a user if imaging conditions are inadequate to improve the quality and efficiency of the counting system (machine translation, paragraphs [0089] and [0100]).
Regarding claim 11, Chen as modified by Shiraishi teaches a program (machine translation, page 5, paragraph under “Example 1” of Chen) configured to cause one or more processors to perform the fish counting method (machine translation, page 5, paragraph under “Example 1” of Chen) according to claim 10 (refer to analysis of claim 10 above).
Regarding claim 14, the combined teachings of Chen as modified by Shiraishi teaches the fish counting system according to claim 1, and further teaches wherein the imaging condition is abnormal further based on a luminance (machine translation, paragraphs [0094]-[0098] of Shiraishi).
Regarding claim 16, Chen as modified by Shiraishi teaches the fish counting system according to claim 1, and Chen further teaches wherein the first area (8-1, fig. 2) has a rectangular shape (fig. 2).
Regarding claim 17, Chen as modified by Shiraishi teaches the fish counting system according to claim 16, but does not explicitly teach wherein the second area has a U-shape.
It is well settled, however, that where the only difference between the prior art and claimed invention is a mere change in shape of a component is generally recognized as being within the level of ordinary skill in the art, the claimed invention is not patentably distinct from the prior art unless a new and unexpected result is produced. MPEP 2144.04 citing In re Rose, 105 USPQ 237 (CCPA 1955). In this case, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen as modified by Shiraishi such that the second area has a U-shape, in order to allow for better coverage of areas where fish may to cluster or congregate based on the structure of the imaging system setup or path that the fish are able to travel.
Regarding claim 18, Chen as modified by Shiraishi teaches the fish counting system according to claim 4, and Chen further teaches wherein:
the first area has a rectangular shape (8-1; fig. 2); and
the third area has a rectangular shape (9; fig. 2), but does not explicitly teach that the second area has a U-shape.
It is well settled, however, that where the only difference between the prior art and claimed invention is a mere change in shape of a component is generally recognized as being within the level of ordinary skill in the art, the claimed invention is not patentably distinct from the prior art unless a new and unexpected result is produced. MPEP 2144.04 citing In re Rose, 105 USPQ 237 (CCPA 1955). In this case, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen as modified by Shiraishi such that the second area has a U-shape, in order to allow for better coverage of areas where fish may to cluster or congregate based on the structure of the imaging system setup or path that the fish are able to travel.
Regarding claim 19, Chen as modified by Shiraishi teaches the fish counting system according to claim 18, but does not explicitly teach wherein three sides of the third area are in contact with the second area.
It is well settled, however, that where the only difference between the prior art and claimed invention is a mere change in shape of a component is generally recognized as being within the level of ordinary skill in the art, the claimed invention is not patentably distinct from the prior art unless a new and unexpected result is produced. MPEP 2144.04 citing In re Rose, 105 USPQ 237 (CCPA 1955). In this case, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen as modified by Shiraishi such that three sides of the third area are in contact with the second area, in order to allow for better coverage of areas where fish may to cluster or congregate based on the structure of the imaging system setup or path that the fish are able to travel.
Regarding claim 20, Chen as modified by Shiraishi teaches the fish counting system according to claim 19, and Chen further teaches wherein a single side of the third area (9) is in contact with the first area (8-1; fig. 2).
Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Shiraishi as applied to claim 2 above, and further in view of Kodaira et al. (JP 2016165238 A) as provided on the IDS, hereafter referred to as “Kodaira” (see the machine translation provided by the applicant).
Regarding claim 7, Chen as modified by Shiraishi teaches the fish counting system according to claim 2, and Chen further teaches the labeling unit (machine translation, page 7, S21-S22) tracking fish based on the moving direction and calculating the change vector of geometric center of the fish in previous frames (machine translation, page 7, S21), but does not explicitly teach:
wherein the labeling unit is configured to:
estimate a position of the fish in an image of interest based on positions of a third fish in past images N and N+1 images before the image of interest, N being a natural number not less than 2, and
assign the same label as to the third fish to a fourth fish that satisfies the estimated fish position and a second tracking enabling condition among fish in the image of interest.
Kodaira teaches a fish counting system (10, abstract, figs. 1-13) including a labeling unit (machine translation, paragraphs [0024]-[0026], [0032]-[0033]) configured to:
estimate a position of the fish in an image of interest based on positions of third fish in past images N and N+1 images before the image of interest, N being a natural number not less than 2 (fig. 11, machine translation, paragraphs [0129]-[0138], [0030]), and
assign the same label as to the third fish to fourth fish that satisfies the estimated fish position and a second tracking enabling condition among fish in the image of interest (machine translation, paragraphs [0032]-[0033]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen as modified by Shiraishi to include the labeling unit configured to estimate a position of the fish in an image of interest based on positions of third fish in past images N and N+1 images before the image of interest, N being a natural number not less than 2, and assign the same label as to the third fish to fourth fish that satisfies the estimated fish position and a second tracking enabling condition among fish in the image of interest, as taught by Kodaira, in order to further improve the tracking of fish as comparing the predicted location of a fish based on subsequent frames would further improve the accuracy of the tracking model.
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Chen in view of Shiraishi as applied to claim 14 above, and further in view of Kurane (JP 2007194687 A), hereafter referred to as “Kurane.”
Regarding claim 15, Chen as modified by Shiraishi teaches the fish counting system according to claim 14, but does not explicitly teach wherein the luminance includes a saturation state.
Kurane teaches a system (see attached machine translation, paragraphs [0009] and [0077]) in which luminance includes a saturation state (machine translation, paragraph [0067], [0087]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen as modified by Shiraishi such that the luminance includes a saturation state, as taught by Kurane, in order to further effectively distinguish poor imaging conditions (machine translation, paragraph [0087] of Kurane).
Claims 1 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (CN 106022459 A), hereafter referred to as “Chen” in view of Roy et al. (WO 2019210421 A1), hereafter referred to as “Roy” and Spampinato et al. (“DETECTING, TRACKING AND COUNTING FISH IN LOW QUALITY UNCONSTRAINED UNDERWATER VIDEOS”), hereafter referred to as “Spampinato.”
Regarding claim 1, Chen discloses a fish counting system (figs. 1-7) comprising:
an image acquisition unit (2, 3) configured to acquire a plurality of images by imaging over time an imaging area (6, 7, 8-1, 8-2, 9, and region of 1, fig. 1, which is captured by 2) in which fluid including a fish flows (figs. 1, 4-7, in which the system exists underwater);
an extraction unit configured to extract a fish for each image (machine translation, page 6, S1-S14); and
a counting unit configured to count a number of fish (machine translation, page 7, S2-S22); and
wherein:
the imaging area includes a first area (8-1, fig. 2) and a second area (8-2, fig. 2), and
the counting unit is configured to count the number of fish when the fish in the first area move to the second area (machine translation, page 7, S22), but does not explicitly teach:
a reporting unit configured to determine that an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal based on a variance of a flatness of the at least a single image exceeding a first threshold or a variance of a turbidity of the fluid in the at least a single image exceeding a second threshold, wherein: the variance of the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image.
Roy teaches a fish imaging system (abstract, figs. 1A-19) including a reporting unit configured to determine that an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal (paragraph [0062]).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen to include a reporting unit configured to determine that an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal, as taught by Roy, in order to promptly notify a user if imaging conditions are inadequate to improve the quality and efficiency of the counting system.
Spampinato teaches a system (abstract, pages 1-6) including a unit configured to determine that an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal based on a variance of a flatness of the at least a single image (pages 3-4).
It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the system of Chen in view of Roy, such that the imaging condition is abnormal based on a flatness of the at least a single image, as taught by Spampinato, in order to improve the detection of image quality (pages 3-4 of Spampinato).
Regarding claim 12, Chen as modified by Roy and Spampinato teaches the fish counting system according to claim 1, and Spampinato further teaches that flatness is determined based on at least one of a reflection of the sun, a reflection of a fish net, or a reflection of a floating object depicted in the at least a single image (pages 3-4 teaching the analysis of the flatness of the image would be capable of detecting irregularities due to reflections of the sun, fish net, or floating object).
Response to Arguments
Applicant's arguments filed 07/21/2025 have been fully considered but are not persuasive.
Applicant argues, on pages 2-3 of the Remarks, that the combined teachings of Chen and Shiraishi do not teach “an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal based on a variance of a flatness of the at least a single image exceeding a first threshold or a variance of a turbidity of the fluid in the at least a single image exceeding a second threshold, wherein: the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image, wherein: the variance of the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image.” Specifically, applicant argues that Shiraishi at paragraph [0091] of the machine translation teaches that the rabidity is determined by assessing the killifish shadow against the background image, not the actual fish against the background.
The examiner respectfully disagrees. While paragraph [0091] of the machine translation may use the word “shadow”, paragraphs [0093]-[0102] of the machine translation makes it apparent that the contrast between the fish shadow of the medaka fish and the background is referring to the silhouette of the fish itself as it appears in the image, not a shadow cast by lighting. Paragraphs [0009] and [0039] of the machine translation explicitly states that the lighting means is situated in a manner that casts no shadow within the space, preventing inaccurate imaging of the fish and determination of the water quality. Paragraphs [0093]-[0102] discuss utilization of brightness correction of the image between the fish silhouette and the background. As shown above, Chen in view of Shiraishi teaches each and every limitation of claims 1 and 10.
Applicant argues, on pages 1-3 of the Remarks, that the combined teachings of Chen and Roy and Spampinato do not teach “an imaging condition is abnormal based on at least a single image, the imaging condition is abnormal based on a variance of a flatness of the at least a single image exceeding a first threshold or a variance of a turbidity of the fluid in the at least a single image exceeding a second threshold, wherein: the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image, wherein: the variance of the turbidity is determined based on a contrast ratio between a fish in the at least a single image and a background in the at least a single image.” Specifically, applicant argues that Spampinato an image condition determination based on a variance of flatness, but an equation for smoothness that includes flatness as a variable.
The examiner respectfully disagrees. Spampinato teaches determining a condition based on a variance of flatness (pages 3-4). While Spampinato may focus on a smoothness parameter, this parameter is based on relative flatness.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Jessica Byun whose telephone number is (571) 272-3212. The examiner can normally be reached Monday - Friday, 9:00 AM - 5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. Agendas may be sent to HaeRie.Byun@uspto.gov.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Peter Poon can be reached on (571) 272-6891. The fax phone number for the organization where this application or proceeding is assigned is (571) 273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/H.J.B./Examiner, Art Unit 3643
/PETER M POON/Supervisory Patent Examiner, Art Unit 3643