DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Claims 3 and 8 specify that “the light source is provided at a suitable place in the containment basin” (emphasis added). The Figures, at best, only show a containment basin 12 and a light source 14 in a configuration where the light source is provided beneath the containment basin, not in or inside of it. Therefore, the light source being located inside the containment basin must be shown or the features canceled from the claims. No new matter should be entered.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The abstract of the disclosure is objected to because: “This invention utilizes” (first sentence) and “This technological innovation” (last sentence) are redundant and already implied by the nature of the abstract being provided, “In comparison to traditional manual counting...” is a comparison of the invention to the prior art, and the last sentence refers to purported merits or speculative applications of the invention. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b) for guidelines for the preparation of patent abstracts.
The disclosure is objected to because the Figures are not accurately described. As an example, paragraph 20 provides, “This light source 14 is positioned at an appropriate location in the containment basin 12, as illustrated in FIG. 1. The light source 14 is a waterproof backlight panel, placed at the bottom of the containment basin 12.” The light source 14 cannot be both inside the containment basin 12 and beneath it. As illustrated, the light source 14 is not “in” the containment basin 12.
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f):
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f). The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) except as otherwise indicated in an Office action.
This application includes claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) because the claim limitations use a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
“a mobile photography device ... a user can control the camera module through the mobile photography device to take a photo of the containment basin, then transmit the photo to the AI database through the wireless transmission unit of the mobile photography device for counting computation of the aquatic fry” in claims 1-5 and “the mobile photography device is equipped with an application program capable of communicating with the AI database; the application program is configured to perform photography, upload photos, count the quantity of the aquatic fry in the photo, and cumulatively tally the quantity of the aquatic fry across multiple photos” in claim 4.
“a wireless transmission unit ... transmit the photo to the AI database through the wireless transmission unit of the mobile photography device for counting computation of the aquatic fry” in claims 1-5.
“a mobile photography device; step 3: uploading the photo via a wireless transmission unit of the mobile photography device to an AI database” in claims 6-10 and “the mobile photography device is preloaded with an application program capable of communicating with the AI database; the application program is configured to perform photography, upload photos, count the quantity of the aquatic fry in the photo, and cumulatively tally the quantity of the aquatic fry across multiple photos” in claim 9.
“uploading the photo via a wireless transmission unit of the mobile photography device to an AI database” in claims 6-10.
A “mobile device” is a portable generic computer. Having wireless capability and a camera is insufficient for performing the control operations of the “mobile photography device”. The operations could, for example, be performed locally by a processor of the device or by a remote processor or a combination, which suggests that the phrase “mobile photography device” uses “device” as a generic placeholder for different devices portable camera-equipped electronic device having a display and wireless connectivity.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f), they are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f), applicant may: (1) amend the claim limitations to avoid them being interpreted under 35 U.S.C. 112(f) (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitations recite sufficient structure to perform the claimed function so as to avoid them being interpreted under 35 U.S.C. 112(f).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1-10 are rejected under 35 U.S.C. 112(b) as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor regards as the invention.
Claims 1 and 6 recite an “AI database”, claim 1 is directed to an “AI counting system” and claim 6 is directed to an “AI counting method”. Ascertaining the meaning of the claimed inventions is complicated by the inclusion of the term “AI” throughout the claims. Simply appending the term “AI” to generic subject matter like a “system”, a “method” or a “database” does not impart any particular technological meaning to the claims. Anything that can be implemented with a computer can be used as part of an “AI” process or “AI” system.
Artificial Intelligence covers a broad and diverse collection of principles and architectures, and generally speaking, refers to the development of computers and/or computer systems that are designed to perform tasks that typically require human intelligence and/or are designed to mimic human-level intelligence in terms of the response to a query requiring such intelligence. AI can also mimic human behaviour but do so at a vastly accelerated rate. There is nothing in the claims, besides perhaps the pre-trained training models, that is at least somewhat related to artificial intelligence, but no more so had the words “deep learning”, “machine learning”, “computer vision”, “pattern recognition method”, “image analysis algorithm” or the like, been used instead of “AI”. This begs the question of what exactly about the “AI” database, method and system is the human behaviour being modelled by the computer system and how is that being accomplished, because there is nothing in the claims to differentiate an “AI” database, method or system from the same but for the inclusion of the term “AI”, for purposes of applying prior art. Accordingly, the term “AI” is interpreted to encompass tools used in AI systems and methods such as machine learning tools like Support Vector Machines (SVM). Dependent claims 2-5 and 7-10 are rejected for inheriting and not curing the deficiencies of claims 1 and 6.
Claims 1 and 6 recite, “wherein the AI database is pre-trained with a large number of various types of aquatic fry photos for counting, forming a plurality of training models” (emphasis added). The term “large” is a relative term and is subjective, which does not clearly delineate the claimed subject matter of a “number of various types of aquatic fry photos” such that a person reading the claim would know what number or range of numbers means they infringe the claims and what number means they do not. In other words, the “large number” being claimed makes the scope of claims 1 and 6 unclear. Additionally, it is unclear whether the claims are describing a current or prior process of creating the “training models” . It is unclear if the claims require training models or if the models are entirely pre-trained. For purposes of applying prior art, “a large number” is interpreted as “a plurality”, the models are considered pre-trained using images of different types of fish photos, and “forming” the models is considered to refer to how they were constructed during the pre-training. Dependent claims 2-5 and 7-10 are rejected for inheriting and not curing the deficiencies of claims 1 and 6.
Claim 1 recites, “the AI database is ... capable of transmitting a data on a quantity of the aquatic fry to the mobile photography device; wherein ... a user can control the camera module through the mobile photography device to take a photo of the containment basin, then transmit the photo to the AI database through the wireless transmission unit of the mobile photography device for counting computation of the aquatic fry; wherein ... the AI database can automatically send the data back to the mobile photography device through the wireless transmission unit and display the data on the display screen; wherein the AI database can cumulatively tally the quantity of the aquatic fry from multiple photos” (emphasis added). Claims 4 and 9 recite, “an application program capable of communicating with the AI database” (emphasis added). Claim 6 recites, “wherein the AI database can rapidly count the quantity of the aquatic fry in the photo, and by repeating the aforementioned four steps, the AI database can cumulatively tally the quantity of the aquatic fry across multiple photos and display a total number of the aquatic fry on the display screen of the mobile photography device.” (emphasis added). The terms “capable of” and “can” imply optionality, meaning the claimed subject matter that follows is not required, but merely a possibility. If the optional subject matter is not required, then it begs the question of why it is included. Any generic computer is “capable of” or “can” perform any of claimed functions or steps, which under the BRI of the claims are merely program instructions executed by a processor to implement image processing and return a result. For purposes of applying prior art, the terms “capable of” and “can” are interpreted to mean what follows is optional and not required to be explicitly taught by the prior art, but merely capable of it. Dependent claims 2-5 and 7-10 are rejected for inheriting and not curing the deficiencies of claims 1 and 6.
Claims 3 and 8 recite, “at a suitable place in the containment basin” (emphasis added). The term “suitable” is subjective and does not clearly delineate the claimed subject matter of the “place in the containment basin” such that a person reading the claim would know what a “suitable” place is compared to an unsuitable place in the containment basin. In other words, the “suitable place” being claimed makes the scope of claims 3 and 8 unclear. Turning to the disclosure, the Figures do not show an embodiment where the light source is contained within or inside of the basin. Paragraph 11 provides, “The light source can be provided at the bottom or side of the containment basin. In a preferred embodiment, the light source is placed at the bottom of the containment basin.” Paragraph 11 seems to provide examples of a “suitable” place, but it is unclear how they describe such a place in the containment basin. The preferred embodiment is shown in Figures 1, 3 and 4. In each Figure, for purposes of applying prior art, “at a suitable place in the containment basin” is interpreted as meaning the light source is located at (near) the basin such that the light serves to illuminate the fish for image capture. Dependent claims 5 and 10 are rejected for inheriting and not curing the deficiencies of claims 3 and 8.
Claims 3 and 8 recite, “a light source and the light source is provided at a suitable place in the containment basin”. As explained above, it is unclear whether applicant intended “in” to mean “inside” or “beneath”. Placing a light source inside a container of water would presumably require some sort of waterproofing and the claims do not recite such subject matter. For purposes of applying prior art, claims 3 and 8 are interpreted as meaning the light source is provided near the containment basin such that the light travels through the basin to illuminate the fish. Dependent claims 5 and 10 are rejected for inheriting and not curing the deficiencies of claims 3 and 8.
Claim 6 recites, “wherein the AI database can rapidly count the quantity of the aquatic fry in the photo” (emphasis added). The term “rapidly” is a relative term and is subjective, which does not clearly delineate the claimed subject matter of “count the quantity of the aquatic fry in the photo” such that a person reading the claim would know what counting rate or speed means they infringe the claim and what value means they do not. For purposes of applying prior art, “rapidly count” is interpreted as “count”. Dependent claims 7-10 are rejected for inheriting and not curing the deficiencies of claim 6.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed inventions are directed to a judicial exception without significantly more.
[Claim 1] An aquatic fry AI counting system for counting various types of aquatic fry, comprising:
(a) a mobile photography device, wherein the mobile photography device comprises at least one (a1) camera module, (a2) a wireless transmission unit, and (a3) a display screen;
(b) a containment basin, wherein the containment basin is used for (b1) holding the aquatic fry to be counted; and
(c) an AI database, wherein the AI database is pre-trained with a large number of various types of aquatic fry photos for counting, forming a plurality of training models for multiple types of aquatic fry, and is capable of (c1) transmitting (c2) a data on a quantity of the aquatic fry to the mobile photography device;
(d) wherein the containment basin contains the aquatic fry to be counted, and (d3) a user can control the camera module through the mobile photography device to (d1) take a photo of the containment basin, then (d2) transmit the photo to the AI database through the wireless transmission unit of the mobile photography device for counting computation of the aquatic fry;
(e) wherein the data of the quantity of the aquatic fry is (e1) generated by (e5) using the corresponding (e2) training model from the photo, and the AI database can automatically (e3) send the data back to the mobile photography device through the wireless transmission unit and (e4) display the data on the display screen;
(f) wherein the AI database can cumulatively tally the quantity of the aquatic fry from multiple photos.
[Claim 6] An aquatic fry AI counting method for counting various types of aquatic fry, comprising:
step 1: (~b1) placing the aquatic fry to be counted in (b) a containment basin;
step 2: (d1) taking a photo of the aquatic fry contained in the containment basin using a (a1) camera module of (a) a mobile photography device;
step 3: (~d2) uploading the photo via (a2) a wireless transmission unit of the mobile photography device to (c) an AI database, wherein the AI database is pre-trained through a large number of various types of aquatic fry photos for counting, forming a plurality of training models for multiple types of aquatic fry, and then (~e5) selecting the (e2) training model corresponding to the type of the aquatic fry being counted to process the photo and (e1) generate (c2) a data on a quantity of the aquatic fry in the photo; and
step 4: (e3) sending the data back to the mobile photography device by the AI database and (e4) displaying the data on the quantity of the aquatic fry in the photo on (a3) a display screen of the mobile photography device;
(g) wherein the AI database can rapidly count the quantity of the aquatic fry in the photo, and by repeating the aforementioned four steps, the AI database can cumulatively tally the quantity of the aquatic fry across multiple photos and display a total number of the aquatic fry on the display screen of the mobile photography device.
Claim Interpretation
Under the broadest reasonable interpretation, the terms of the claims are presumed to have their plain meaning consistent with the specification as it would be interpreted by one of ordinary skill in the art. See MPEP 2111.
Based on the plain meaning of the words in the independent claims, the broadest reasonable interpretation of claim 1 is a fish fry counting system comprising a mobile device with a camera that is controlled by a user to transmit images to a remote computer having a database for calculating an accumulated total of fry and returning the value to the mobile device’s display. The broadest reasonable interpretation of claim 6 is a method of using the mobile device.
The device of limitation (a) is recited at a high level of generality and the claims do not put much limitation on it beyond it being portable having wireless connectivity, being able to control a camera, upload images, receive data from a server and display image and/or video data, which applies to essentially every modern consumer device with a wireless transceiver (e.g., Bluetooth, Wi-Fi, cellular), display, battery, processor, memory and operating system, e.g., smartphones, tables, laptops, and desktops.
Limitation (c) includes the term “AI”, which also appears in the preambles, though no patentable weight is given to terms in a preamble that are not later referenced in the body of the claim or otherwise required by the body of the claim. Though explained in more detail in the rejection of claims 1 and 6 under 35 U.S.C. 112(b) above, the term “AI” as used in the pending claims is used more as a general link to an area of computer science than it is as a limiting description of any particular technical elements. A commonality in the various definitions and uses of the term “AI” is that they generally require a computer system, one or more of some type of model that is trainable or has been pre-trained, often one or more machine learning algorithms, and are designed to mimic or repeat tasks that require a human level of intelligence (which is somewhat subjective to clearly define) at either the same proficiency or better and at the same rate or higher. However, neither the claims nor the specification offer any substantive technological disclosure that amounts to a purported improvement of an AI system, or a use of known prior art AI processes or AI-specific architectures applied in a particular way. Instead, as stated by applicant, “The main technical method of the present invention involves the use of a mobile photography device, wherein the mobile photography device can be a smartphone, tablet, or smart camera.” Specification at par. 9.
Paragraphs 2-6 of the specification identify some prior approaches to counting fish fry and acknowledges problems and needs in the prior art such as the need to automate manual counting to alleviate the labor-intensive nature of the task. The description that follows paragraphs 2-6 describes embodiments of using mobile devices to capture images of fish fry and offload the processing to a server for calculating a running total that is sent back to the devices for display. However, the extent of “AI” in the claims amounts to merely labeling the database as an “AI database” that is associated with “a plurality of training modules for multiple types of aquatic fry”. In other words, as far as AI is concerned, limitations (a)-(g) amount to implementing the method with a database over a network where the database (implemented by a remote generic computer) is capable of implementing AI models or AI-specific tasks, but none of which are required by the claims. A database is typically stored electronically in a physical device having some kind of storage, like a server, but can also be a type of data, e.g., a table. Given the context of the claims, the former interpretation is most reasonable.
The “training models” of limitation (c) do not require any specific AI or Machine Learning (ML) algorithm or technique.
Limitation (g) on its face is entirely optional in claim 6, as the phrases “can rapidly count” and “by repeating ... can cumulatively tally” amount to a possibility if further unrecited steps are performed. Even if limitation (g) was not optional, it essentially amounts to repeating the same steps but keeping track of the total number of fish from all of the images, which is also not a unique concept to AI or Machine Learning. In short, under the broadest reasonable interpretation of claims 1 and 6, an explicit reference to Artificial Intelligence is not within the scope of the claims.
Limitations (e), (e1), (e2), (e3) and (e4) are broadly set forth and could reasonably describe a person manually selecting a pre-existing model, where limitation (e3) specifies that at least some part of the communication between the database the mobile device is performed by a computer even if the transmission is initiated by a person.
Step 1: do the claims fall within any statutory category?
Claim 1 recites a system comprising a mobile photography device, which comprises a camera module, a wireless transmission unit and a display screen, and therefore, is a machine. Claim 6 recites a series of steps and therefore, is a process. See MPEP 2106.03. (Step 1: YES).
Step 2A, Prong One: do the claims recite a judicial exception?
Overall, the subject matter of the independent claims describes counting the number of a particular type of fish in a plurality of images and accumulating the total across all images. Limitations (a), (a1), (a2) and (a3) amount to a generic mobile device with wireless connectivity. The basin of limitation (b) is utilized for its expected and typical function of containing the fish suspended in water while they are counted. Limitation (c) amounts to a generic remote processing computer, e.g., a server. Limitations (d1) and (d2) describe common functionality of smart phones and other mobile photography devices. Limitations (e), (e1), (e2), (e3) and (e4) describe choosing an existing model trained for the type of fish being counted which is automatically sent to the mobile device, e.g., as either a user-generated request or an (implicit) automatically or programmatically-generated request from the mobile device to the database. A user is explicitly recited in limitation (d1) and the act of “selecting” a model in limitation (~e5) is reasonably interpreted as a user-based action (by-hand), made by a person who interacts with a database using a mobile device.
The recitation of structures in the claims, e.g., “database” (which implies electronic storage), “training models” (which require a physical storage location to be useable), “mobile photography device”, “camera module”, “wireless transmission unit” and “display screen” does not negate the manual or mental aspects of the claims addressed above because the claims merely use the structural components as tools to carry out the aims and decisions of the user of the database in their efforts to use the appropriate counting model for the fish they recognize. See MPEP 2106.04(a)(2), subsection III.C.
Under the broadest reasonable interpretation, the claims describe the observations, evaluations, judgments, and opinions of a person counting the total number of fish in one or more images captured by the user using the mobile device, which falls within the mental process grouping of abstract ideas. See MPEP 2106.04(a)(2), subsection III. (Step 2A, Prong One: YES).
Step 2A, Prong Two: Do the claims as a whole integrate the recited judicial exception into a practical application of the exception?
Claim 1 recites additional elements: “mobile photography device, wherein the mobile photography device comprises at least one camera module, a wireless transmission unit, and a display screen”, “containment basin ... used for holding the aquatic fry to be counted”, “AI database”, “a plurality of training models”, “transmitting a data on a quantity of the aquatic fry to the mobile photography device” “transmit the photo to the AI database through the wireless transmission unit of the mobile photography device”, “the data of the quantity of the aquatic fry is generated by using the corresponding training model from the photo” and “send the data back to the mobile photography device through the wireless transmission unit and display the data on the display screen”.
Claim 6 recites additional elements “uploading the photo via a wireless transmission unit of the mobile photography device to an AI database”, “a plurality of training models”, “process the photo and generate a data on a quantity of the aquatic fry in the photo”, “sending the data back to the mobile photography device”, “displaying the data on the quantity of the aquatic fry in the photo on a display screen of the mobile photography device” and “display a total number of the aquatic fry on the display screen of the mobile photography device.”
The additional elements describe generic computer components and/or operations thereof (e.g., using a camera-equipped tablet or smartphone to capture and store fish fry images, search the Internet for a suitable model to tally the number of detected objects) recited at a high level of generality and do not amount to any of the relevant considerations for evaluating whether additional limitations integrate a judicial exception into a practical application provided in MPEP 2106.04(d), subsection I. The additional elements amount to merely including instructions to implement the abstract idea in a client-server type of architecture that connects the user with a remote repository of stored models to peruse and select a suitable model for the specific object being counted in an image collection, thereby merely using a computer (mobile device and implied computer of the database) as a tool to perform the abstract idea. See MPEP 2106.05(f). Thus, the additional elements of the claims invoke computers merely as a tool to perform an existing process: counting the number of like objects (or animals) in an image collection. See MPEP 2106.05(f)(2).
The additional elements of “an AI database” and “training models” that are “pre-trained” with images of the same object, merely confine the use of the abstract idea to a particular technological environment (ML/AI model selection and execution) and thus fail to add an inventive concept to the claims. See MPEP 2106.05(h).
It should be noted that because the courts have made it clear that mere physicality or tangibility of an additional element or elements is not a relevant consideration in the eligibility analysis, the physical nature of any underlying hardware in the additional elements, e.g., “mobile photography device”, “database”, “camera”, “wireless transmission” and “display” does not affect this analysis. See MPEP 2106.05(I).
Dependent claims 2 and 7 describe the type of material of the “containment basin” as being “light-transmitting”, which is common practice for containers that hold fish.
Dependent claims 3 and 8 further specify a light source, which is nearly universally part of every smartphone and common to mobile photography devices in general. Generally speaking, adding a light source to illuminate an object being imaged by a camera is merely using a common mobile device to capture images using the flash function, setting the flash light source to be constant or using the display itself as the light source and a front-facing camera to capture images instead of a rear-facing camera.
Dependent claims 4 and 9 merely append the concept that the mobile device uses an application, which has been a staple of smartphone operating systems since the first iPhone and is the typical data structure POSITA would expect to be used with a mobile device that communicates with a database, besides perhaps a browser.
Dependent claims 5 and 10 merely append that the mobile device uses an application, which has been a staple of smartphone operating systems since the first iPhone and is the typical data structure POSITA would expect to be used with a mobile device that communicates with a database, besides perhaps a browser.
The additional elements do not improve the functioning of a computer. See MPEP 2106.04(d)(1). The additional elements merely append common functionality to the recited physical components without substantially impacting the judicial exception within the claims.
The specification’s background section sets forth a technical problem when discussing prior approaches to fish fry counting: “Considering factors like the transparent body color of the aquatic fry, their overlapping positions, and continuous movement, these methods fail to meet the needs of aquaculture farmers both domestically and internationally. Consequently, manual labor remains the predominant method used” (par. 6). Thus, the disclosed embodiments provide a platform for hosting and executing computer vision models made available for user selection, e.g., by using their smartphone, which is no different than a generic computer system operated over a network.
Whether evaluated individually or in combination, the additional elements do not integrate the recited judicial exception into a practical application and the claims, therefore, are directed to the judicial exception and do not integrate the recited judicial exception into a practical application of the exception. (Step 2A, Prong Two: NO).
Step 2B: do the claims as a whole amount to significantly more than the judicial exception?
As explained with respect to Step 2A Prong Two, the additional elements of the pending claims amount to performing the abstract idea using a computer as a tool to perform an existing process, fish fry counting with computer vision and pre-trained models, which cannot provide an inventive concept. See MPEP 2106.05(f). Based on the high-level of specify of the technical aspects of the independent and dependent claims as compared to subject matter in the specification, including the drawings, that is in certain aspects more specific and rooted in the technical problems being solved in the additional elements, the additional elements do not constitute an improvement to the functioning of a computer or to another technology because they represent what is well-understood, routine, conventional activity. See MPEP 2106.04(d)(1). As an example, see Recent Advances Of Deep Learning Algorithms For Aquacultural Machine Vision Systems With Emphasis On Fish to Li et al., particularly Fig. 1 and section 3.
Dependent claims 2-5 and 7-10 do not add any particular subject matter that is specific to Deep Learning, Machine Learning or Artificial Intelligence, let alone a specific application thereof. These claims either add routine features in the field of fish fry counting like using transparent fish tanks and light sources, or describe features common to mobile devices like executing applications or “apps” to communicate with more complex software executed at a remote location, e.g., cloud computing. The claims do not recite any specific training algorithm or complex decision logic for automatically selecting the best model based on a specific input image, for example. The claims merely link the abstract idea, generally, to a particular field of use: Artificial Intelligence, which in itself is difficult in many contexts to fully and clearly distinguish from Machine Learning and applications of computer vision. See MPEP 2106.05(h). The claims do not any technical features that are any more specific to “AI” than they are to the field of computer vision as a whole. Additionally, the dependent claims do not include any subject matter that amounts to an improvement of the functioning of a computer, or an improvement to any other technology or technical field, as they merely describe and apply known concepts in the fields of image analysis and machine learning. See MPEP 2106.04(d)(1).
The additional elements in combination with the judicial exception do not provide an improvement to the functioning of a computer or any other technology or technical field. Even considering each claim as a whole, the claims do not amount to significantly more than the recited judicial exception and fail to encompass an inventive concept (Step 2B: NO). Claims 1-10 therefore, are not eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed inventions absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-10 are rejected under 35 U.S.C. 103 as being unpatentable over TW Pat. No. I687159 to Chen et al. (hereinafter “Chen” - citations refer to the attached machine translation from Espacenet) in view of U.S. Pat. Appl. Pub. No. 20180263223 to Kodaira et al. (hereinafter “Kodaira”) and in further view of Automated Fish Fry Counting and Schooling Behavior Analysis Using Computer Vision to Labuguen et al. (hereinafter “Labuguen”).
Regarding claim 1, Chen teaches an aquatic fry AI counting system for counting various types of aquatic fry, comprising:
a mobile photography device (The central computing device 14 comprises a signal receiving unit 141, central computing unit 142, and display module 143. See Chen at Fig. 2. The image capture device 13 is a camera. See Chen at Fig. 1. Together, image capture device 13, computing device 14 and database 15 form a mobile photography device because such a device may be moved to a location for installation. If it is moveable, it is portable or mobile.), wherein the mobile photography device comprises at least one camera module (Chen, pg. 1, “an image capture device is installed in ... the waterway and continuously capture multiple images, where the images cover at least the first detection area and the gate detection line”), a wireless transmission unit (Chen, pg. 4, “the central computing device 14 is connected to the image capturing device 13 in a wired or wireless manner to receive the image captured by the image capturing device 13.”), and a display screen (Chen, pg. 9, “If the central computing device 14 is equipped with a display module (the display module 143 shown in FIG. 2), the central computing device 14 may further display the fry type on the display module 143 (step S36), so that Users know. Furthermore, the central computing device 14 may also only record the fish fry types without directly displaying them, or transmit the fish fry types to remote equipment or portable devices for display via the network, without limitation.”);
a containment basin (fry tank 11. See Chen at Fig. 1), wherein the containment basin is used for holding the aquatic fry to be counted (Chen, pg. 6, “The counting system 1 also includes a fish collecting tank 16 corresponding to the outlet 122 of the one-way water channel 12. The fry 2 flows from the fry water tank 11 to be tested into the one-way water channel 12, and finally from the water outlet 122 of the one-way water channel 12. Flow into the fish collecting tank 16. When a fry 2 flows into the fish collecting tank 16, it means that the fry 2 once appeared in the image captured by the image capturing device 13, and has been analyzed by the central counting device 14, meaning that the central computing device 14 has counted This fry is 2 (ie, the number of fry has been +1).”); and
an AI database (database 15. See Chen at pg. 5. fry species table 151 and fry size table 152 form a database. See Chen at pg. 10. The database is an “AI database” because it is used with machine learning tools including Support Vector Machines (SVM). See Chen at pg. 10), wherein the AI database is pre-trained with a large number of various types of aquatic fry photos for counting (Chen, pg. 9, “the central computing device 14 queries the pre-trained fish fry type table 151 in the database 15 according to the obtained object characteristics (step S32), and confirms the fish fry type corresponding to the object according to the query result (step S34). That is, the central computing device 14 can determine the type of fish fry in the image by querying the fish fry type table 151.”), forming a plurality of training models for multiple types of aquatic fry (Chen, pg. 10, “In a training phase (for example, when manufacturing counting system 1), the manufacturer can incorporate dozens of common fish fry species in the aquaculture industry into the above-mentioned feature extraction algorithm to pre-train a set of models that can classify fry (ie, Establish the fry species table 151 and fry size table 152).”), and is capable of transmitting data of the aquatic fry to the mobile photography device (Chen, pg. 10, “under normal circumstances, the central computing device 14 will count the number of fry to be 1 when the tracked fry 2 passes through the gate detection line 124 of the one-way waterway 12; in special circumstances (such as multiple fry 2 Overlap or side by side), the central computing device 14 will count the number of fry according to the number of overlap or side by side of the fry 2. Plural fry 2 side by side can easily be identified one by one during foreground separation and marked separately, but overlapping plural fry 2 may be marked as single fry 2.”);
wherein the containment basin contains the aquatic fry to be counted (Chen, pg. 6, “The fry 2 flows from the fry water tank 11 to be tested into the one-way water channel 12, and finally from the water outlet 122 of the one-way water channel 12.”), and transmit the photo to the AI database through the wireless transmission unit of the mobile photography device (Chen, pg. 4, “the central computing device 14 is connected to the image capturing device 13 in a wired or wireless manner to receive the image captured by the image capturing device 13.”, pg. 10, “central computing device 14 obtains an image from the image capturing device 13”) for counting computation of the aquatic fry (Chen, pg. 9, “In the aforementioned step S30 (or step S16 in FIG. 3), the central computing device 14 mainly uses a feature extraction algorithm to obtain one or more features of the object from the image”);
wherein the data of the aquatic fry is generated by using the corresponding training model from the photo (Chen, pg. 10, “the manufacturer can incorporate dozens of common fish fry species in the aquaculture industry into the above-mentioned feature extraction algorithm to pre-train a set of models that can classify fry (ie, Establish the fry species table 151 and fry size table 152).”, pg. 11, “Through the calculation of the number of centroids, the number of fry counted by the central computing device 14 can be matched with the actual number of fry passing through the gate detection line 124, thereby greatly reducing the counting error.”), and the AI database can send the data back to the mobile photography device through the wireless transmission unit and display the data on the display screen (Chen, pg. 5, “the central computing device 14 will not only record and accumulate the total number of fry passing through all one-way water channels 12, but also separately record and accumulate the number of fry passing through each one-way water channel 12 for the user’s reference”, pg. 9, “transmit the fish fry types to remote equipment or portable devices for display via the network”, pg. 10, “the central computing device 14 can directly display the fry size on the display module 143 (step S46), or transmit the fry size to a remote device or portable device via the network for display”); and
wherein the AI database can cumulatively tally the quantity of the aquatic fry from multiple photos (Chen, pg. 7, “If it is determined that the counting operation does not need to be stopped, the counting system 1 returns to step S10, and the image capturing device 13 continuously captures multiple images, and the central computing device 14 continuously performs image analysis and counts the number of fry. If it is determined that the counting operation needs to be stopped, the counting system 1 ends the counting method of the present invention.”), but does not teach that which is explicitly taught by Kodaira.
Kodaira teaches a user can control the camera module through a mobile photography device (e.g., communication interface 350, measurement image acquisition unit 311a, and output unit 331a. See Kodaira at FIG. 13) to take a photo (Kodaira, par. 45, “The measurement image acquisition unit 11 acquires, over time, n measurement images of a region to be measured in a passage region where a fluid containing fish passes through. ... Examples of the imaging unit include: ... camera-equipped mobile terminals such as a camera-equipped mobile phone, a camera-equipped smartphone, and a camera-equipped tablet terminal; a web camera-equipped computer; and a camera-equipped head-mounted display.”, par. 145, “The display unit can be, for example, the above-mentioned monitor. The display unit displays, for example, the measurement images acquired by the measurement image acquisition unit, fish position information acquired by the fish position information acquisition unit, predicted fish position information acquired by the predicted fish position information acquisition unit, the information on the same fish identified by the same fish identification unit, and information on a fish count obtained by the fish counting unit. The display unit may display the information acquired in the first modification to the seventh modification. The number of pieces of information to be displayed is not limited to particular values and may be, for example, one or more or all.”);
a database (a server that stores an organized collection of data is an electronic database) capable of transmitting a data on a quantity of the aquatic fry to the mobile photography device (Kodaira, par. 193, “The fish counting system sends n measurement images acquired using the measurement image acquisition unit 311 a in the place X to the server 370, and fish are counted in the server 370 to obtain a fish count. The obtained fish count is then output with the output unit 311 a.”); and
the database can automatically send the data back to the mobile photography device (Kodaira, par. 193, “The fish counting system sends n measurement images acquired using the measurement image acquisition unit 311 a in the place X to the server 370, and fish are counted in the server 370 to obtain a fish count. The obtained fish count is then output with the output unit 311 a.”).
Chen discloses an image-based counting system where fish contained in a tank are photographed as they leave their tank and travel down a channel that passes by a camera. The camera continuously captures images that are processed by a separate, central computing device. Thus, Chen shows that it was known in the art before the effective filing date of the claimed invention to allocate processing of images acquired by a camera-equipped mobile device to a database to execute image analysis programs, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, alleviating the need for manual counting. Kodaira discloses a mobile terminal in communication with a server for counting fish where the server processes images of fish from the mobile device and the obtained fish count is sent to the display of the terminal. Kodaira also indicates that the disclosed system is compatible with a cloud computing arrangement (par. 190). Thus, Kodaira shows that it was known in the art before the effective filing date of the claimed invention to offload processing of counting fish fry images to a server that provides the count to a display of a terminal or mobile photography device, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, alleviating the need for manual counting.
A person of ordinary skill in the art would have been motivated to combine the smartphone, terminal-server functionality and/or cloud computing environment disclosed by Kodaira with the system of Chen, to thereby re-locate the main functionality of the central computing unit 14 and database 15 to a cloud server in communication with the camera of the battery-operated mobile photography device that executes an application to count fish by sending captured images to the server which returns the count value and other related information. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of reducing the required computational resources of the mobile photography device.
Chen in view of Kodaira does not teach that which is explicitly taught by Labuguen.
Labuguen teaches taking a photo of a containment basin made of a light-transmitting material (Figure 2 shows a clear, glass containment basin. See Labuguen at pg. 256. A light source is provided along with a diffuser to increase contrast between the fish and the background. See Labuguen at section III.A. The upper and lower glass sections along with the diffuser form a containment basin. See Labuguen at Figure 2. In that sense, the light source is “in” the containment basin.).
Chen and Kodaira are analogous to the claimed invention for the reasons provided above. Labuguen discloses an image-based counting system where fish contained in a tank are photographed by a camera and then counted. Thus, Labuguen shows that it was known in the art before the effective filing date of the claimed invention to orient a camera used for fish counting to include a tank containing the fish in the camera’s field of view, which is analogous to the claimed invention in that it is pertinent to the problem being solved by the claimed invention, alleviating the need for manual counting.
A person of ordinary skill in the art would have been motivated to combine the image processing software, light source, diffuser, camera, tank material and tank arrangement disclosed by Labuguen with the system disclosed by Chen in view of Kodaira, to thereby capture images and count fish in multiple configurations including fish traveling down a tube past a camera and fish swimming around their tank. Based on the foregoing, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have made such modification according to known methods to yield the predictable results to have the benefit of increasing the number of environments in which the system can be deployed.
Regarding claim 2, Chen in view of Kodaira and in further view of Labuguen teaches the aquatic fry AI counting system of claim 1, wherein the containment basin is made of a light-transmitting material (Figure 2 shows a clear, glass containment basin. See Labuguen at pg. 256).
The rationale for obviousness is the same as provided for claim 1.
Regarding claim 3, Chen in view of Kodaira and in further view of Labuguen teaches the aquatic fry AI counting system of claim 1, wherein the aquatic fry AI counting system further comprises a light source and the light source is provided at a suitable place in the containment basin (Figure 2 shows a clear, glass containment basin that has a lower compartment containing lights. See Labuguen at pg. 256. The lower section is part of the containment basin, and therefore, the light and the diffuser are in the basin. See Labuguen at section III.A).
The rationale for obviousness is the same as provided for claim 1.
Regarding claim 4, Chen in view of Kodaira and in further view of Labuguen teaches the aquatic fry AI counting system of claim 1, wherein the mobile photography device is equipped with an application program capable of communicating with the AI database (Kodaira at par. 187, “the units installed apart from one another can be controlled centrally and remotely operated.”);
the application program is configured to perform photography, upload photos (Kodaira, par. 187, “sends n measurement images acquired using the measurement image acquisition unit 311 a in the place X to the server 370, and fish are counted in the server 370 to obtain a fish count. The obtained fish count is then output with the output unit 311 a.”), count the quantity of the aquatic fry in the photo (Kodaira, par. 187, “fish are counted in the server 370 to obtain a fish count.”), and cumulatively tally the quantity of the aquatic fry across multiple photos (See Kodaira at Fig. 11).
The rationale for obviousness is the same as provided for claim 1.
Regarding claim 5, Chen in view of Kodaira and in further view of Labuguen teaches the aquatic fry AI counting system of claim 3, wherein the light source is electrically connected to a mobile power supply (Camera phones and smart phones, by their nature of being mobile devices, are equipped with rechargeable batteries. See Kodaira at par. 45, “a camera-equipped mobile phone, a camera-equipped smartphone”).
The rationale for obviousness is the same as provided for claim 1.
Claim 6 substantially corresponds to claim 1 by reciting a method of using the system of claim 1. Claim 6 recites “uploading” instead of “transmit”, which has no patentable distinction in the cloud-computing context disclosed by Kodaira at paragraph 190. Claim 6 recites “selecting the training model” which corresponds to “using the corresponding training model” in claim 1. Claim 6 further recites “repeating the aforementioned steps”, which is equivalent to the continuous processing disclosed by Chen at page 7. Claim 6 further recites, “display a total number of the aquatic fry on the display screen of the mobile photography device” which corresponds to the “send the data back” clause of claim 1, but further adding that the quantity is a total, which corresponds to “cumulatively tally” in claim 1. See Chen at pg. 5. Claims 7-10 substantially correspond to claims 2-5 by each reciting the method of using the system of claims 2-5 respectively.
The rationale for obviousness is the same as provided for claim 1.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN P POTTS whose telephone number is (571)272-6351. The examiner can normally be reached M-F, 9am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN P POTTS/Examiner, Art Unit 2672
/SUMATI LEFKOWITZ/Supervisory Patent Examiner, Art Unit 2672
Prosecution Insights