DETAILED ACTION
Status of the Claims
Original claims 1-10 filed December 11, 2023, are pending.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Claim Objections
Claim(s) 1, 6, and 7 is/are objected to because of the following informalities:
In claim 1, line 3 on page 15, “HMI” should be replaced with “human-machine interface (HMI)”
In claim 1, line 5 on page 15, “human-machine interface (HMI)” should be replaced with “HMI”
In claim 6, line 22 on page 16, “HMI” should be replaced with “human-machine interface (HMI)”
In claim 7, line 5 on page 17, “human-machine interface (HMI)” should be replaced with “HMI”
Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
the image reception unit of claim 6,
the image analysis unit of claim 6,
the data storage unit of claim 6, and
the artificial intelligence training unit of claim 8.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-10 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 1 recites “generating an image analysis result by analyzing the first image based on an artificial intelligence learning model configured to generate an analysis result by analyzing an image” (lines 7-8 on page 15). This limitation is functional at least because it recites a function of generating an image analysis result and computer-implemented at least because it requires using artificial intelligence.
MPEP 2161.01, Subsection I, provides instructions for determining whether there is adequate written description for a computer-implemented functional claim limitation. It explains that “the specification must describe the claimed invention in a manner understandable to a person of ordinary skill in the art in a way that shows that the inventor actually invented the claimed invention at the time of filing” and that “[p]roblems satisfying the written description requirement for original claims often occur when claim language is generic or functional, or both.” Generic claim language may be problematic when the specification does not describe a sufficient number of species to demonstrate possession of a full genus. Functional claim language may be problematic when an algorithm for performing the function is not described, such that one of ordinary skill in the art would not understand how the inventor intended the claimed function to be performed. The claim limitation noted above suffers from both of these types of problems.
First, “generating an image analysis result” is incredibly broad and generic. It covers any kind of determination, estimation, prediction, etc. that can be made from an image. This would include, for example, determining whether a particular object or component is present or absent, a location of an object within an image, recognizing an object in an image, classifying an image as a whole, reading text in an image, determining a type of image, making a future prediction based on an image, etc. Examiner has reviewed the specification, but has not identified any specific examples of an “image analysis result.” Instead, the specification merely repeats the generic term “image analysis result”. For example, par. [0043] (all citations are to the published application, unless otherwise noted) merely states that “an image analysis result may be generated by analyzing the first image based on an artificial intelligence learning model configured to generate an analysis result by analyzing an image (S220).” In view of the lack of any specific examples of an image analysis result, the written description is not sufficient to demonstrate to one of ordinary skill in the art that the inventor had possession of the full scope of the claimed invention at the time of filing. Therefore, the claim does not comply with the written description requirement of 35 U.S.C. 112(a).
Second, the specification does not describe any algorithm for generating an image analysis result. While the specification, like the claims, states that the result is “based on an artificial intelligence learning model”, there is no explanation of any specific sequence of steps taken to generate an image analysis result. The claim recites a desired result (i.e., generation of an image analysis result), but does not explain how the inventor intended the function to be performed. Note that while the claim recites the analysis result is “based on an artificial intelligence learning model”, there is no explanation of details of the model (e.g., its inputs, outputs, or architecture), nor how the generation of the image analysis result is based on the artificial intelligence learning model. For example, does the AI learning model perform image pre-processing, such as noise reduction or super-resolution, before some other steps produce the analysis result? Or, does the AI learning model directly accept the received image as an input and directly produce the image analysis result? If so, are any other inputs received and what is the format of the output? None of these questions are answered by the specification. Stating that a result is achieved “based on” use of a broad class of tools (i.e., artificial intelligence learning models) does not explain what specific sequence of steps is taken to achieve that result. In view of the lack of explanation of how the image analysis result is generated, the written description is not sufficient to demonstrate to one of ordinary skill in the art that the inventor had possession of the full scope of the claimed invention at the time of filing. Therefore, the claim does not comply with the written description requirement of 35 U.S.C. 112(a).
Claims 6 and 7 recite similar limitations that also lack adequate written description for substantially the same reasons. Claims 2-5 and 8-10 are also rejected at least because they include the limitations of at least one of claims 1, 6, and 7.
Claim 2 is further rejected because it further recites “a first artificial intelligence learning model configured to generate a determination result” and this limitation also suffers from substantially the same problems discussed above with respect to claim 1. I.e., “determination result” is incredibly broad, yet the specification provides no specific examples, and there is no description of an algorithm for how the determination result is generated. Claim 2 is also further rejected because it further recites “a second artificial intelligence learning model configured to generate the image analysis result”, which lacks sufficient written description for substantially the same reasons explained above with respect to claim 1. Claim 8 recites similar limitations and therefore is further rejected for substantially the same reasons as claim 2. Claims 3-5 and 9-10 are further rejected at least because they include the limitations of claims 2 or 8.
Claim 4 recites “displaying a reflection ratio for each item starting from items having a greatest influence on the image analysis result, and displaying modifiable items among the items” at lines 11-12 on page 16. This limitation is functional at least because it recites a function of displaying information and computer-implemented at least because it apparently involves controlling a computer display.
Examiner has reviewed the specification in search of a description of how this function is performed, but has found none. In general, the specification simply restates the language used in the claims. See, e.g., Fig. 5, step S510, and pars. [0012], [0018], and [0060]. There is no explanation of what a reflection ratio is or how it is acquired, determined, estimated, etc. so that it can be displayed. There also is no explanation of what the items are, what influence they have on the image analysis result, or how it can be determined which item has a greatest influence on the image analysis result. There also is no explanation of what makes an item “modifiable” or how it can be modified. The specification plainly does not provide an algorithm for performing the claimed function noted above. In view of the lack of explanation in the specification, the written description is not sufficient to demonstrate to one of ordinary skill in the art that the inventor had possession of the full scope of the claimed invention at the time of filing. Therefore, the claim does not comply with the written description requirement of 35 U.S.C. 112(a).
Claim 4 further recites “displaying, when a first item, which is one of the items, is selected by the user, samples of the image analysis result in which a reflection ratio for the first item is changed” at lines 13-14 on page 16. This limitation also lacks adequate written description for substantially the same reasons discussed above. I.e., the specification generally restates the language of the claim without any explanation of an algorithm for how the function is performed. For example, there is no explanation of how an item is selected, what the samples are, how the samples are displayed, how the reflection ratio is changed, how a change in the reflection ratio produces or influences different samples, etc. In view of the lack of explanation in the specification, the written description is not sufficient to demonstrate to one of ordinary skill in the art that the inventor had possession of the full scope of the claimed invention at the time of filing. Therefore, the claim does not comply with the written description requirement of 35 U.S.C. 112(a).
Claim 10 recites similar limitations and also lacks adequate written description for substantially the same reasons. Claims 5 is also rejected at least because it includes the limitations of claim 4.
Claims 6 and 8 recite means-plus-function limitations that have been found to be indefinite under 35 U.S.C. 112(b) based on failure of the specification to disclose structure, material or act that performs the entire claimed function. See rejections below. “A means- (or step-) plus-function limitation that is found to be indefinite under 35 U.S.C. 112(b) based on failure of the specification to disclose corresponding structure, material or act that performs the entire claimed function also lacks adequate written description.” MPEP 2181, Subsection IV. Therefore, claims 6 and 8 are also rejected for failing to comply with the written description requirement of 35 U.S.C. 112(a). Claims 7 and 9-10 are also rejected at least because they include the limitations of at least one of claims 6 and 8.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim(s) 1-10 is/are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 1 recites “a camera” at line 4 on page 15, “an image camera” (emphasis added) at line 6 on page 15, “receiving a first image” (emphasis added) at line 5 on page 15 and “generating an image analysis result by analyzing the first image based on an artificial intelligence learning model configured to generate an analysis result by analyzing an image” (emphasis added) at lines 7-8 on page 15. There are multiple issues of indefiniteness regarding these limitations.
First, it is unclear whether “an image camera” at line 6 refers to the “camera” recited at line 4, or some other camera. On the one hand, use of the article “an” suggests that a new claim element is being introduced and the label “image” is added, which suggests that “an image camera” is different from the earlier-recited camera. On the other hand, both cameras apparently capture images of an HMI, which suggests that they may be the same. This ambiguity makes the scope of the claim unclear and renders the claim indefinite.
Second, it is unclear whether “an image” at line 8 refers to the “first image” or some other image. On the one hand, use of the article “an” suggests that a new claim element is being introduced and the label “first” is omitted, which suggests that “an image” is different from the earlier-recited first image. On the other hand, the claim earlier recites “analyzing the first image” and it is unclear what other image would be analyzed, which suggests that “an image” is referring to the earlier-recited first image. This ambiguity makes the scope of the claim unclear and renders the claim indefinite.
Third, it is also unclear whether “an analysis result” at line 8 refers to the “image analysis result” recited at line 7, or some other analysis result. On the one hand, use of the article “an” suggests that a new claim element is being introduced and the label “first” is omitted, which suggests that “an analysis result” is different from the earlier-recited image analysis result. On the other hand, the claim earlier recites that the image analysis result is generated based on the artificial intelligence learning model that apparently also generates the “analysis result”, which suggests that the “analysis result” may be the same as the earlier-recited “image analysis result”. This ambiguity makes the scope of the claim unclear and renders the claim indefinite.
Claims 2-5 are also indefinite at least because they include the indefinite limitations of claim 1. Claim 6 recites similar limitations to those in the first issue discussed above with respect to claim 1, and is also indefinite for substantially the same reasons. Claim 7 further recites limitations similar to those in the second and third issues discussed above with respect to claim 1, and is also indefinite for substantially the same reasons. Claims 8-10 are also indefinite at least because they include the indefinite limitations of claims 6 and 7.
Claim 4 recites “displaying a reflection ratio” at line 11 of page 16. It is unclear what is meant by this term. “Reflection” is the bouncing back of a wave from a surface. For example, light bounces off a mirror, forming a reflection. A “ratio” is a comparison of two quantities. However, the claimed invention does not have any apparent connection to reflections, much less ratios of reflections. I.e., there is no discussion of any waves or reflection of waves in the specification, much less ratios of such reflections. There is also no evidence of record showing that “reflection ratio” would be understood to have a different ordinary meaning by one of ordinary skill in the art. Therefore, it appears that the phrase “reflection ratio” is being used contrary to its ordinary meaning.
Where applicant acts as his or her own lexicographer to specifically define a term of a claim contrary to its ordinary meaning, the written description must clearly redefine the claim term and set forth the uncommon definition so as to put one reasonably skilled in the art on notice that the applicant intended to so redefine that claim term. Process Control Corp. v. HydReclaim Corp., 190 F.3d 1350, 1357, 52 USPQ2d 1029, 1033 (Fed. Cir. 1999). Examiner has reviewed the specification, but has not identified any clear redefinition of the term “reflection ratio”. Instead, the specification generally repeats the same language used in the claims. The term “reflection ratio” is indefinite because the specification does not clearly redefine the term.
Claim 10 recites the same term and is also indefinite for substantially the same reasons as claim 4. Claim 5 is also indefinite at least because it includes the limitations of claim 4.
Claim 4 recites the limitation "each item" in line 11 on page 16 and “the items” in line 12 on page 16. There is insufficient antecedent basis for these limitations in the claim. There is no prior introduction of any items in the claims, so it is unclear what “each item” is referring to. Furthermore, it is unclear whether “the items” refers to whatever set of items that is referenced by “each item” or refers to the apparently smaller subset of “items having a greatest influence on the image analysis result”.
Claim 4 also recites “a reflection ratio” at line 14 of page 16 and it is unclear whether this refers to the same reflection ratio introduced at line 11, or a different reflection ratio.
Claim 4 also recites “a first image analysis result” at line 15 on page 16 and it is unclear whether this refers to “an image analysis result” in claim 1 at line 7 on page 15, “an analysis result” in claim 1 at line 8 on page 15, or some other image analysis result.
For at least these reasons, claim 4 is indefinite.
Claim 10 recites similar limitations and is also indefinite for substantially the same reasons as claim 4. Claim 5 is also indefinite at least because it includes the limitations of claim 4.
Claim 6 recites an image reception unit, an image analysis unit, and a data storage unit. Claim 8 recites an artificial intelligence training unit. All of these units are being interpreted under 35 U.S.C. 112(f) – see Claim Interpretation above.
“A rejection under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph may be appropriate in the following situations when examining means-plus-function claim limitations under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(1) when it is unclear whether a claim limitation invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(2) when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph is invoked and there is no disclosure or there is insufficient disclosure of structure, material, or acts for performing the claimed function; and/or
(3) when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph is invoked and the supporting disclosure fails to clearly link or associate the disclosed structure, material, or acts to the claimed function.” MPEP 2181, Subsection III.
Examiner has reviewed the specification in an attempt to identify corresponding structure, material, or acts for the claimed functions of the units interpreted under ‘112(f). The specification does describe an image reception unit, an image analysis unit, a data storage unit, and an artificial intelligence training unit at Fig. 7 and pars. [0070]-[0076] (as-published). However, the description generally restates what is already recited in the claims without reciting any structure for performing those functions. For example, [0072] states that “the image reception unit may be configured to receive a first image obtained by capturing a screen of a human-machine interface (HMI) from the image camera,” but does not describe any structure that performs this function. Accordingly, situation (2) described in the MPEP (see quotation above) appears to be present, which renders the claims indefinite.
The specification does describe several components of a general computer system ([0079] and Fig. 8), but does not clearly state which (if any) of the components are used to implement the different units of Fig. 7, or explain how such an implementation would be accomplished. Therefore, even if the computer components could be seen as structure for perform the functions of the claimed units, they are not clearly linked as in situation (3) described in the MPEP, which renders the claim indefinite.
Furthermore, even if the units are understood to be computer-implemented, then “the specification must disclose an algorithm for performing the claimed specific computer function, or else the claim is indefinite under 35 U.S.C. 112(b).” MPEP 2181, Subsection II.B. As explained above in the rejections under 35 U.S.C. 112(a), the specification does not disclose algorithms for performing at least some of the functions of the claimed units, which would further render the claim indefinite. For example, there is no description of an algorithm “to generate an image analysis result by analyzing the image based on an artificial intelligence learning model” – see above.
Claims 7 and 9-10 are also indefinite at least because they include the limitations of at least one of claims 6 and 8.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-10 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea judicial exception without significantly more.
The patent subject matter eligibility (SME) test is set forth in MPEP 2106. It includes multiple steps, sub-steps, and prongs.
Step 1
Claims 1-5 are to a process and claims 6-10 are apparently to a machine or manufacture.
Step 2A, Prong One
Claim 1 recites “receiving a first image obtained by capturing a screen of a human-machine interface (HMI) from an image camera,” which can be seen as a mental process because a human can look at and capture a mental image of a screen of an HMI. See MPEP 2106.04(a)(2), Subsection III. Note that the phrase “from an image camera” can be seen as requiring performance of the mental process in a computer environment or using a computer as a tool to perform the mental process, but neither preclude the claim from being considered to recite a mental process. Id at further subsection C.
Claim 1 further recites “generating an image analysis result by analyzing the first image,” which can be seen as a mental process because a human can analyze an image and produce some sort of evaluation, judgment or opinion as an analysis result. Claims 2-4 and 8-10 recite similar limitations.
Claim 1 further recites “storing collection data corresponding to the first image based on the image analysis result,” which can be seen as a mental process because a human can mentally remember an evaluation, judgment or opinion as an image analysis result. Alternatively, or additionally, the human could write their mental analysis result down on a sheet of paper in order to store it. Note that use of physical aids such as pencil and paper does not preclude a claim limitation from reciting a mental process. Id. at further subsection B.
Claims 6 and 7 recite limitations similar to those in claim 1.
Claims 2, 3, 4, 8, 9, and 10 recite training, updating and/or retraining an artificial intelligence learning model. Especially at the level of generality recited in the claims, this can be seen as a mental process of learning (or re-learning) how to perform a task or make a certain judgement. While the claims require artificial intelligence, this merely requires that the mental process is performed in a computer environment or by using a computer as a tool, neither of which preclude the claim from being considered to recite a mental process. Id at further subsection C.
Claims 2 and 8 further recite generating a determination result by determining information displayed on the HMI in the first image, which can be seen as a mental process because a person can view an HMI and mentally determine what information is being displayed.
Step 2A, Prong Two
None of the additional elements recited in the claims integrate the abstract idea into a practical application because none go beyond:
Merely reciting the words "apply it" (or an equivalent) with the judicial exception, or merely including instructions to implement an abstract idea on a computer, or merely using a computer as a tool to perform an abstract idea, as discussed in MPEP § 2106.05(f);
Adding insignificant extra-solution activity to the judicial exception, as discussed in MPEP § 2106.05(g); and
Generally linking the use of a judicial exception to a particular technological environment or field of use, as discussed in MPEP § 2106.05(h).
Claim 1 recites that an image is captured from a camera, but this amounts to mere instructions to implement an abstract idea on a computer, the camera being a computer component used merely as a tool to perform an existing process of visually perceiving a screen. The same analysis applies to recitations of a camera in claims 6 and 7. Claim 1 further recites that an analysis result is generated based on an artificial intelligence learning model, but this also amounts mere instructions to implement an abstract idea on a computer at least because it serves to simply invoke computers merely as a tool to perform an existing process of analyzing what is being displayed on a screen. The same analysis applies to the artificial intelligence learning models recited in claims 2 and 6-8.
Claim 1 recites storing collection data. This is mere data gathering, which is a type of insignificant extra-solution activity. Claims 6 and 7 recite similar limitations. Claims 2-5 and 8-10 recite various interfaces for receiving data and displaying other information. Especially at the level of generality used in the claims, this is also mere data gathering insignificant extra-solution activity. The displaying generally presents options to a user and the interfaces generally receive a selection or modification input from the user. The selection or modification inputs are used to optimize (i.e., train, re-train, and/or update) a model. This is similar to, for example, presenting offers to potential customers and gathering statistics generated based on testing about how potential customers responded to the offers, where the statistics are then used to calculate an optimized price, which was previously found to be insignificant extra-solution activity. MPEP 2106.05(g), citing to OIP Technologies, 788 F.3d at 1363, 115 USPQ2d at 1092-93. Claim 5 recites providing the collection data to the user through an integrated dashboard, which is insignificant extra-solution activity at least because it amounts to necessary outputting – the purpose of collecting the information is so that it can be provided to a user.
Claim 1 recites that the captured image is of an HMI screen. Claims 6 and 7 recite similar limitations. These limitations do not more than generally link the use of the abstract idea to a particular field of use or technological environment, similar to how Limiting the abstract idea of collecting information, analyzing it, and displaying certain results of the collection and analysis to data related to the electric power grid was found to be simply an attempt to limit the use of the abstract idea to a particular technological environment. MPEP 2106.05(h), citing to Electric Power Group, LLC v. Alstom S.A., 830 F.3d 1350, 1354, 119 USPQ2d 1739, 1742 (Fed. Cir. 2016).
Step 2B
The analysis to be performed in Step 2B of the SME is outlined in MPEP 2106.05, Subsection II. Identification of the additional element(s) in the claim from Step 2A Prong Two is carried over. Conclusions from Step 2A Prong Two are carried over.
Any additional elements considered to be insignificant extra-solution activity are re-evaluated for whether they are well-understood, routine, and conventional. Storing collection data, as recited in at least claims 1, 6 and 7, covers receiving or transmitting data over a network, electronic recordkeeping, and storing and retrieving information in memory, all of which have been recognized by the courts as well-understood, routine, and conventional activity. MPEP 2106.05(d), Subsection II. The displaying recited in claims 4-5 and 10, especially at the level of generality used, is analogous to presenting offers. For example, the display in claim 4 is used to offer items to a user for selection. Such presenting offers has been found by the courts to be well-understood, routine, and conventional activity. Id. The interfaces recited in claims 3 and 9 are recited at a high level of generality and function to receive information from a user, but such receipt of information is well-understood, routine, and conventional activity. Id. Claim 5’s requirement that collected data is presented through an integrated dashboard is general and broad enough to cover transmission of data over a network, presentation of offers, and/or gathering of statistics, all of which are well-understood, routine and conventional activity. Id. None of the additional elements, or combinations of elements, recited in the claims are other than what is well-understood, routine, conventional activity in the field, or simply append well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception.
Conclusion
Claims 1-10 are rejected under 35 U.S.C. 101 because they recite an abstract idea and none of their additional elements integrate the abstract idea into a practical application or otherwise amount to significantly more than the abstract idea.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-10 is/are rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by ‘Walli’ (US 2022/0300759 A1).
Regarding claim 1, Walli discloses a method for collecting data through HMI screen recognition and image analysis by using a camera (e.g., Figures 3-4), the method comprising:
receiving a first image (e.g., Fig. 4, step 404) obtained by capturing a screen of a human-machine interface (HMI) (e.g., [0052], Fig. 3, vehicle dashboard HMI; also see, e.g., [0088]) from an image camera (e.g., Fig. 3, camera 302);
generating an image analysis result (e.g., [0088], a reading from a dashboard indicator, such as an odometer reading, fuel level, etc.) by analyzing the first image based on an artificial intelligence learning model (e.g., Fig. 3, region of interest identifier 320 and/or image processing models 342; also see, e.g., [0056], [0060], [0062]) configured to generate an analysis result by analyzing an image (Note the ‘112(b) rejections; e.g., Fig. 3, captured image is processed by ROI identifier 320 to identify ROI, which is then processed by an image processing model 342 to generate a reading; Either or both of the ROI or the reading could be an analysis result; The “image” could be the captured image or specifically the ROI region within that image); and
storing collection data corresponding to the first image based on the image analysis result (e.g., [0054], Fig. 3, telematics system 350 sends vehicle information to data collection system 120; e.g., [0035], “vehicle data collection system 120 is … to receive and store the data collected by the telematics system”).
Regarding claim 2, Walli discloses the method of claim 1, wherein, after the storing of the collection data, the method further comprises training the artificial intelligence learning model by using the collection data (e.g., [0077], [0082], Fig. 6, model may be further trained using feedback received on prior outputs – i.e., the collection data), and the artificial intelligence learning model includes:
a first artificial intelligence learning model configured to generate a determination result by determining information displayed on the HMI in the first image (e.g., [0056]-[0057], Fig. 3, region of interest (ROI) identifier 320 is an AI model that determines what information is displayed on the dashboard HMI image, and where it is located; e.g., Fig. 3 shows determination results with ROIs 304-1 and 304-2); and
a second artificial intelligence learning model configured to generate the image analysis result for the first image based on the determination result and the information (e.g., [0058], Fig. 3, image processing model coordinator 330 selects what image processing model to use for each ROI, given what type of information it has been determined to include; For example, an ROI determined to include a fuel gauge will be assigned a fuel gauge image processing model; e.g., [0060], the image processing model is an AI model that processes the ROI to extract a reading, such as an odometer reading, as an image analysis result).
Regarding claim 3, Walli discloses the method of claim 2, wherein the training of the artificial intelligence learning model includes:
providing a first interface, which is capable of receiving an input of first data modification information for the determination result from a user (e.g., Fig. 6, [0079], display of predicted region of interest 616 and interface used to provide feedback and/or edits; also see, e.g., Fig. 5), to a user terminal (e.g., [0078], Fig. 6, the device including display component 612);
generating the image analysis result for the first image based on the first data modification information for the determination result received through the first interface (e.g., [0077], [0082], the feedback regarding the ROI is used to further train the models, which generate the image analysis result; e.g., Fig. 6 shows a display including the image analysis result with modification data);
providing a second interface, which is capable of receiving an input of second data modification information for the image analysis result from the user (e.g., Fig. 6, [0080], display of predicted label 618 and value 619 and interface used to provide feedback and/or edits), to the user terminal (e.g., [0078], Fig. 6, the device including display component 612);
performing retraining of the first artificial intelligence learning model and the second artificial intelligence learning model based on the first data modification information for the determination result received through the first interface and the second data modification information for the image analysis result received through the second interface (e.g., [0077], [0082], the feedback regarding the ROI, the label, and the value is used to further train the models); and
updating the first artificial intelligence learning model and the second artificial intelligence learning model through the retraining (Training (or re-training) a model necessarily includes updating its parameters).
Regarding claim 4, Walli discloses the method of claim 3, wherein, after the providing of the second interface, the method further comprises:
displaying a reflection ratio for each item starting from items having a greatest influence on the image analysis result, and displaying modifiable items among the items (Note that the meaning of this limitation is unclear – see ‘112(b) rejection above; As best understood in the view of the indefiniteness, the speedometer reading 619 in Fig. 6 can be considered a “reflection ratio” at least because the speed reading value reflects a ratio of the speedometer traversed by the needle; For example, if the speedometer includes values from 0 to 100 miles per hour and the needle has moved across a ratio of half the speedometer, then that reflects a speed reading of 50 MPH; This can be considered to have the greatest influence on the image analysis result, since the image analysis result is the reading of the speedometer);
displaying, when a first item, which is one of the items, is selected by the user, samples of the image analysis result in which a reflection ratio for the first item is changed (e.g., Fig. 6, [0080], if speedometer value – i.e., a reflection value as best understood – is selected and changed, then the new value is displayed as a sample); and
receiving a first image analysis result selected by the user among the samples, and the performing of the retraining includes performing retraining of the second artificial intelligence learning model based on the first image analysis result (e.g., [0080], [0077], [0082], whatever new value/sample is input by the user is used to retrain the models).
Regarding claim 5, Walli discloses the method of claim 4, wherein, after the storing of the collection data, the method further comprises providing the collection data to the user through an integrated dashboard (e.g., [0077]-[0078], Fig. 6 can be considered an “integrated dashboard” that provides previously-stored collection data to the user for review; e.g., [0054], [0035], collection data is sent to vehicle data collection system 120, which is an integrated dashboard that provides “a telematics service, including live tracking, record keeping, and reporting services to end user (client) devices”).
Regarding claim 6, Examiner notes that the claim recites an apparatus comprising various “units” that perform functions substantially similar to functions recited in claim 1. These units are interpreted under 35 U.S.C. 112(f), but their scopes are unclear because the specification does not clearly disclose their corresponding structures – see above. Walli discloses the functions of claim 1 (see above). Walli further discloses various structures for performing these functions, such as camera interface 315 and image processing unit 310 of Fig. 3. Therefore, as best understood in view of the indefiniteness, it appears that Walli’s disclosure also falls within the scope of claim 6 for substantially the same reasons presented in the rejection of claim 1.
Regarding claim 7, Examiner notes that the claim depends from claim 6 and recites limitations that are substantially the same as limitations recited in claim 1. Walli discloses the inventions of claims 6 and 1 – see above. Accordingly, claim 7 is also rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Walli for substantially the same reasons as those presented in the rejections of claims 6 and 1.
Regarding claim 8, Examiner notes that the claim depends from claim 6 and recites limitations that are substantially the same as limitations recited in claim 2. Walli discloses the inventions of claims 6 and 2 – see above. Accordingly, claim 8 is also rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Walli for substantially the same reasons as those presented in the rejections of claims 6 and 2.
Regarding claim 9, Examiner notes that the claim depends from claim 6 and recites limitations that are substantially the same as limitations recited in claim 3. Walli discloses the inventions of claims 6 and 3 – see above. Accordingly, claim 9 is also rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Walli for substantially the same reasons as those presented in the rejections of claims 6 and 3.
Regarding claim 10, Examiner notes that the claim depends from claim 6 and recites limitations that are substantially the same as limitations recited in claim 4. Walli discloses the inventions of claims 6 and 4 – see above. Accordingly, claim 10 is also rejected under 35 U.S.C. 102(a)(1) and 102(a)(2) as being anticipated by Walli for substantially the same reasons as those presented in the rejections of claims 6 and 4.
Conclusion
The following prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
‘Singh’ (“A System Framework for Non-Intrusive Monitoring of HMI States for Detecting Human-in-the-Loop Error Precursors,” 2020)
Describes various approaches for camera-based HMI monitoring
‘Reynolds’ (US 2023/0099239 A1)
Describes processing images of HMIs to perform various analysis of an industrial device or process
‘Al-Husseini’ (US 2022/0180647 A1)
Captures images of a flight instrument panel, analyzes them to extract flight data, then analyzes the flight data
‘Balasubramanian’ (US 2022/0172100 A1)
Includes examples of various user interfaces for providing feedback to guide model training, re-training, etc.
Other examples of using artificial intelligence models to read screens, meters, gauges, and/or other HMIs:
‘Minisankar’ (US 2023/0057340 A1)
‘Ha’ (US 2023/0045188 A1)
‘Taki’ (US 2022/0309611 A1)
‘Narahari’ (US 2021/0192207 A1)
‘Schwartz’ (US 2021/0174131 A1)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEOFFREY E SUMMERS whose telephone number is (571)272-9915. The examiner can normally be reached Monday-Friday, 7:00 AM to 3:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information a