DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 03/02/2026 and 10/02/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Response to Arguments
Applicant’s amendment filed 03/02/2026 has been entered and made of record. Claim 1 is amended. New Claims 21-39 were added. Claims 2-20 are cancelled. Claims 1 and 21-39 are pending.
Applicant’s arguments with respect to claims 1 and 21-39 have been considered but are moot because the new ground of rejection set forth below. The applicant argues on page 6 of the remarks filed the cited prior art of James et al. US PG-Pub(US 20210142052 A1) does not explicitly teach applying data store rules to the aquatic life images to generate a data curator score for each aquatic life image that reflects an importance of each aquatic image to training of a machine learning model. The Examiner agrees as James doesn’t appear to teach this limitation. However, after further search and consideration, the newly discovered art of Liu et al. (CN 108921058 A) would disclose this limitation. Please see updated claim rejection under 35 USC § 103 below.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim 1 and 21-39 are rejected under 35 U.S.C. 103 as being unpatentable over James et al. US PG-Pub(US 20210142052 A1) in view of Liu et al. (CN 108921058 A).
Regarding Claim 1, James teaches a computer-implemented method comprising: receiving aquatic life images that a user has uploaded through an interface (¶[0012], “In a general aspect, a method includes: receiving, by the one or more processors, first media representative of aquatic cargo”, ¶[0012] discloses receiving aquatic cargo images and ¶[0140] discloses a user interface capable of displaying images.); identifyingbased on the data curator scores(¶[0073], “the image may be dropped if the cropped representation module 120 is unable to detect the three key elements, even if they exist. In some implementations, the cropped representation module 120 may apply a filter to the high resolution image 118 based on a score of the three elements. The filter may drop one or more images if the score of the three elements is below a predetermined threshold”, ¶[0073] discloses filtering images that are below a certain confidence score.); providing the proper subset of the aquatic life images to one or more data annotators(¶[0075], “the cropped representation module 120 applies a photography technique to the image to filter out images below a certain threshold level of sharpness. For example, the photography technique can include one or more techniques for performing a Laplacian transform, applying a Gaussian smoothing filter, a median filter, or a mean filter on the image. By applying the photography technique to the “fish face” image, the system 100 improves its recognition because the image is sharper than before. The sharp “fish face” 122 is then provided to the embedding generation module 124.”, ¶[0075] discloses once the images are filtered out the images are provided to an embedding generation module.);; receiving annotation data generated by the one or more data annotators for the proper subset of the aquatic life images (¶[0092] “The cluster identification module 128 can then transmit a notification to the fish faces database 130 and/or to a client device indicating that a re-identification of a fish has been found. A user can review the notification and analyze the corresponding embedding, the match, recorded media 114, high resolution image 118, and the cropped image 122 to determine whether an actual match exists. If the user determines a match does not exist, the user may provide feedback, through the client device, to the monitoring server 102 or at the monitoring server 102 to fine tune the CNN embedding model to produce the correct embedding.”, discloses allowing a user to determine if the image contains a fish or not and providing feedback to the machine learning model to produce the correct embedding); andtraining the machine learning model using the annotation data and the proper subset of the aquatic life images. ¶[0124] “The system 300 may receive data from a user or from another external party indicating that the mapping did not correctly map to a particular cluster in the high dimensional space, e.g., the embedding mapped to cluster 1 in the high dimensional space rather than mapping to cluster 2. Additionally, the user may indicate the generated embedding was incorrect or the embedding should not have matched to another embedding, when a re-identification process occurred. [0125] “The system 300 may receive additional positive example data with anchors and additional negative example data for fine tuning the trained neural network model 314. The model trainer 306 may use the newly received positive and negative example data to update the trained neural network model 314.”, ¶[0124] discloses determining from the user if the embedding was incorrect and ¶[0125] discloses retraining the system using positive and negative example data to update the machine learning model to generate the appropriate classification.)
James does not explicitly teach applying data store rules to the aquatic life images to generate a data curator score for each aquatic life image that reflects an importance of each aquatic image to training of a machine learning model
Liu teaches applying data store rules to the aquatic life images to generate a data curator score for each aquatic life image that reflects an importance of each aquatic image to training of a machine learning model (Page 3, Paragraph 4, “extracting the image characteristic of the training set fish picture, to generate a characteristic picture, and the characteristic picture is associated with the fish picture; generating network according to the characteristic picture training area, to obtain the probability score of all candidate areas and each candidate area in the characteristic picture; and training the standard model according to the characteristic picture, the candidate area and the probability score of each candidate area.”, as disclosed in this section of the prior art, the features of a fish image are extracted and a confidence score is generated and used to train a machine learning network.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by James with Liu in order to generate a confidence score and train the model based on the score. One skilled in the art would have been motivated to modify James in this manner in order to realize accurate identification of fish information. (Liu, Abstract)
Regarding Claim 21, the combination of James and Liu teach the method of claim 1, where Liu further teaches wherein the importance of each aquatic image is based at least on a confidence associated with the machine learning model. (Page 3, Paragraph 4, “extracting the image characteristic of the training set fish picture, to generate a characteristic picture, and the characteristic picture is associated with the fish picture; generating network according to the characteristic picture training area, to obtain the probability score of all candidate areas and each candidate area in the characteristic picture; and training the standard model according to the characteristic picture, the candidate area and the probability score of each candidate area.”, as disclosed in this section of the prior art, the features of a fish image are extracted and a confidence score is generated and used to train a machine learning network.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by James with Liu in order to generate a confidence score and train the model based on the score. One skilled in the art would have been motivated to modify James in this manner in order to realize accurate identification of fish information. (Liu, Abstract)
Regarding Claim 22, the combination of James and Liu teach the method of claim 1, where Liu further teaches wherein the importance of each aquatic image is based at least on a species of any fish shown in each aquatic image. (Page 7, Paragraph 2-3, “S107, determining the final fish identification model according to the generalization performance score, and identifying fish according to the final fish identification model. Namely, after determining the final fish identification model, it can be fish identification according to the final fish identification model. Specifically, it can obtain fish picture, and the fish picture input final fish identification model, final fish identification model according to fish picture output fish area and corresponding to the fish information. wherein the fish information comprises but not limited to fish type, type probability”, in this section of the prior art, the fish identification model is trained using the probability score of the fish image to determine a type or species of the fish.)
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify the claimed invention as taught by James with Liu in order to use the importance score to generate an identification of the fish. One skilled in the art would have been motivated to modify James in this manner in order to realize accurate identification of fish information. (Liu, Abstract)
Regarding Claim 23, the combination of James and Liu teach the method of claim 1, where James further teaches comprising providing an output of the machine learning model to a device of the user in response to receiving the aquatic life images. (¶[0092] discloses transmitting the output of the machine learning model to the user to review the aquatic image.)
Regarding Claim 24, the combination of James and Liu teach the method of claim 1, where James further teaches comprising discarding any aquatic life images that are not members of the subset. ((¶[0073], “the image may be dropped if the cropped representation module 120 is unable to detect the three key elements, even if they exist. In some implementations, the cropped representation module 120 may apply a filter to the high resolution image 118 based on a score of the three elements. The filter may drop one or more images if the score of the three elements is below a predetermined threshold”, ¶[0073] discloses filtering images that are below a certain confidence score.)
Regarding Claim 25, the combination of James and Liu teach the method of claim 1, where James further teaches wherein the interface comprises an application programming interface (API) (¶[0141], “Embodiments of the invention can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a client computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the invention, or any combination of one or more such back end, middleware, or front end components”, ¶[0141] discloses a back end component linked with the graphical interface.).
Regarding Claim 26, the combination of James and Liu teach the method of claim 1, where James further teaches comprising storing multiple copies of the aquatic life images that are members of the proper subset on different storage devices. (¶0097], “In some implementations, the fish faces database 130 can also store other data associated with the received embedding, the mapped cluster, and the nomenclature. For example, the fish faces database 130 may also provide the recorded media 114, the high resolution image 118, and the square “fish face” 122 associated with a corresponding embedding. By storing photos of fish in the fish faces database based on some criteria, e.g., a typical condition factor for the weight or a fish that has good identification recognition history, then an implementer or designer of the system 100 can generate perception models that may be used to track progression of disease, weight, size, and other criteria of the fish, over time.”, discloses storing multiple fish images pertaining to different type of fish species in a database.)
Regarding Claim 27, the combination of James and Liu teach the method of claim 1, where James further teaches wherein the annotation data includes species identification data. (¶[0092] “The cluster identification module 128 can then transmit a notification to the fish faces database 130 and/or to a client device indicating that a re-identification of a fish has been found. A user can review the notification and analyze the corresponding embedding, the match, recorded media 114, high resolution image 118, and the cropped image 122 to determine whether an actual match exists. If the user determines a match does not exist, the user may provide feedback, through the client device, to the monitoring server 102 or at the monitoring server 102 to fine tune the CNN embedding model to produce the correct embedding.”, discloses allowing a user to determine if the image contains a fish or not and providing feedback to the machine learning model to produce the correct embedding);
Regarding Claim 28, claim 28 is considered an apparatus claim substantially corresponding to claim 1. Please see the discussion of claim 1 above for a discussion of similar limitations. Furthermore, James teaches a system([0025] FIG. 1 is a diagram of an example configuration of a system for identification of a fish within an aquatic structure.) comprising: one or more computer processors, and one or more non-transitory computer-readable media that store instructions which (See ¶[0039]), when executed, cause the one or more computer processors to perform operations (See ¶[0039])
Regarding claim 29, it is substantially similar to claim 21 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 30, it is substantially similar to claim 22 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 31, it is substantially similar to claim 23 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 32, it is substantially similar to claim 34 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 33, it is substantially similar to claim 25 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 34, it is substantially similar to claim 26 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 35, it is substantially similar to claim 27 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding Claim 36, claim 36 is considered an storage medium claim substantially corresponding to claim 1. Please see the discussion of claim 1 above for a discussion of similar limitations. Furthermore, James teaches One or more non-transitory computer-readable media that store instructions which(See ¶[0136]), when executed, cause one or more computer processors to perform operations comprising (See ¶[0136])
37. (New) The media of claim 36, wherein the importance of each aquatic image is based at least on a confidence associated with the machine learning model.
Regarding claim 37, it is substantially similar to claim 21 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 38, it is substantially similar to claim 22 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Regarding claim 39, it is substantially similar to claim 23 respectively, and is rejected in the same manner, the same art, and reasoning applying.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HAN D HOANG whose telephone number is (571)272-4344. The examiner can normally be reached Monday-Friday 8-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JOHN M VILLECCO can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAN HOANG/Primary Examiner, Art Unit 2661
Prosecution Insights