DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Preliminary Amendment
This action is responsive to the preliminary amendments and remarks received 06 March 2025. Claims 1 - 20 are currently pending.
Specification
The lengthy specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Objections
Claim 6 is objected to because of the following informalities: Line 2 of claim 6 recites, in part, “a fault in measured training data” which appears to contain a minor informality. The Examiner suggests amending the claim to --a fault in the measured training data-- in order to maintain consistency with line 2 of claim 5 and to improve the clarity and precision of the claim. Appropriate correction is required.
Claim 9 is objected to because of the following informalities: Lines 2 - 3 of claim 9 recite, in part, “each reflective value associated with a band” which appears to contain a minor informality. The Examiner suggests amending the claim to --each reflective value of the plurality of reflective values associated with a band-- in order to maintain consistency with line 2 of claim 9 and to improve the clarity and precision of the claim. Appropriate correction is required.
Claim 16 is objected to because of the following informalities: Line 2 of claim 16 recites, in part, “a fault in measured training data” which appears to contain a minor informality. The Examiner suggests amending the claim to --a fault in the measured training data-- in order to maintain consistency with line 2 of claim 15 and to improve the clarity and precision of the claim. Appropriate correction is required.
Claim 19 is objected to because of the following informalities: Lines 2 - 3 of claim 19 recite, in part, “each reflective value associated with a band” which appears to contain a minor informality. The Examiner suggests amending the claim to --each reflective value of the plurality of reflective values associated with a band-- in order to maintain consistency with line 2 of claim 19 and to improve the clarity and precision of the claim. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 4, 14, 16 and 20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention because it is unclear as to what is meant by “a YOLOR machine learning model.” The instant specification, see page 8 paragraph 0045 of the instant specification, defines YOLOR as “You Only Look One Representation”. (emphasis added) However, the accepted meaning of a YOLOR machine learning model by the prior art appears to be a “You Only Learn One Representation” (emphasis added) machine learning model, see for example Nien et al. U.S. Publication No. 2025/0366818 A1 paragraph 0009, Szuba et al. U.S. Publication No. 2023/0360402 A1 paragraph 0075 and/or Sherman et al. U.S. Publication No. 2022/0415051 A1 pargraph 0040. Therefore, the Examiner asserts that it is unclear as to how “a YOLOR machine learning model” in claim 4 should be interpreted. Should it be interpreted as a “You Only Look One Representation” machine learning model or a “You Only Learn One Representation” machine learning model? Clarification and appropriate correction are required. For purposes of examination, the Examiner will treat the YOLOR machine learning model as a “You Only Learn One Representation” machine learning model.
Claim 14 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention because it is unclear as to what is meant by “a YOLOR machine learning model.” The instant specification, see page 8 paragraph 0045 of the instant specification, defines YOLOR as “You Only Look One Representation”. (emphasis added) However, the accepted meaning of a YOLOR machine learning model by the prior art appears to be a “You Only Learn One Representation” (emphasis added) machine learning model, see for example Nien et al. U.S. Publication No. 2025/0366818 A1 paragraph 0009, Szuba et al. U.S. Publication No. 2023/0360402 A1 paragraph 0075 and/or Sherman et al. U.S. Publication No. 2022/0415051 A1 pargraph 0040. Therefore, the Examiner asserts that it is unclear as to how “a YOLOR machine learning model” in claim 14 should be interpreted. Should it be interpreted as a “You Only Look One Representation” machine learning model or a “You Only Learn One Representation” machine learning model? Clarification and appropriate correction are required. For purposes of examination, the Examiner will treat the YOLOR machine learning model as a “You Only Learn One Representation” machine learning model.
Claim 16 recites the limitation "The method of claim 15," (emphasis added) in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim 20 recites the limitation "The method of claim 11," (emphasis added) in line 1. There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1 - 5, 7, 9 - 15, 17, 19 and 20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception, an abstract idea, without significantly more. The claims are directed towards determining a pavement condition of a roadway and identifying a pavement management strategy, which is an abstract idea.
The claims recite, at a high level of generality, determining a reflective value corresponding to the portion of the roadway, determining a pavement condition of the portion of the roadway based at least in part upon the reflective value, and identifying a pavement management strategy based least in part upon the pavement condition.
The limitation of “determining a reflective value corresponding to the portion of the roadway”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind using observation, evaluation, judgment, and opinion but for the recitation of generic computer components. That is, other than reciting “a computing device comprising a processor and memory” (see claim 11) and “an application for pavement condition assessment comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to…” (see claim 11) nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the recitation of the aforementioned generic computer components, the claimed determining a reflective value encompasses a user observing imagery of a roadway and performing an evaluation, judgment and/or opinion by mentally identifying a reflective value illustrated in the imagery that corresponds to the roadway. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, with or without the use of a physical aid such as pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP § 2106.04(a)(2)(III).
Similarly, the limitation of “determining a pavement condition of the portion of the roadway based at least in part upon the reflective value”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind using observation, evaluation, judgment, and opinion but for the recitation of generic computer components. That is, other than reciting “a computing device comprising a processor and memory” (see claim 11) and “an application for pavement condition assessment comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to…” (see claim 11) nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the recitation of the aforementioned generic computer components, the claimed determining a pavement condition of the portion of the roadway based at least in part upon the reflective value encompasses a user observing imagery of a roadway and performing an evaluation, judgment and/or opinion by mentally deciding on a condition of the pavement of the roadway in the imagery. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, with or without the use of a physical aid such as pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP § 2106.04(a)(2)(III).
Relatedly, the limitation of “identifying a pavement management strategy based least in part upon the pavement condition”, as drafted, is a process that, under its broadest reasonable interpretation, covers performance of the limitation in the mind using observation, evaluation, judgment, and opinion but for the recitation of generic computer components. That is, other than reciting “a computing device comprising a processor and memory” (see claim 11) and “an application for pavement condition assessment comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to…” (see claim 11) nothing in the claim element precludes the step from practically being performed in the mind. For example, but for the recitation of the aforementioned generic computer components, the claimed identifying a pavement management strategy based least in part upon the pavement condition encompasses a user observing imagery of a roadway, mentally deciding on the condition of the pavement of the roadway in the imagery and performing an evaluation, judgment and/or opinion by mentally deciding on a plan for managing the roadway. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, with or without the use of a physical aid such as pen and paper, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP § 2106.04(a)(2)(III).
This judicial exception is not integrated into a practical application. In particular, the claims recite additional elements of: “obtaining satellite imagery including a portion of a roadway”, “a computing device comprising a processor and memory” and “an application for pavement condition assessment comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to…”.
The limitation of “obtaining satellite imagery including a portion of a roadway” is mere pre-solution activity, data gathering, recited at a high level of generality, and thus is insignificant extra-solution activity. See MPEP § 2106.05(g). In addition, all uses of the recited judicial exception require such data gathering, and, as such, this limitation does not impose any meaningful limits on the claims. This limitation amounts to necessary data gathering. See MPEP § 2106.05.
Further, the limitations of “a computing device comprising a processor and memory” and “an application for pavement condition assessment comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to…” are recited at a high level of generality such that they amount to no more than mere instructions to apply the exception using generic computer components. Furthermore, the claims as a whole merely describe how to generally “apply” the concept of determining a pavement condition of a roadway in a computer environment. Simply implementing the abstract idea on a generic computer is not a practical application of the abstract idea. See MPEP § 2106.05(f).
Even when viewed in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea. Accordingly, the claims are directed to an abstract idea.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional element of: “obtaining satellite imagery including a portion of a roadway” is mere pre-solution activity, data gathering, recited at a high level of generality, and is thus insignificant extra-solution activity. Furthermore, the additional elements of: “a computing device comprising a processor and memory” and “an application for pavement condition assessment comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to…” amount to no more than mere instructions to apply the exception using generic computer components. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept. Even when considered in combination, these additional elements represent mere instructions to implement an abstract idea or other exception on a computer and insignificant extra-solution activity, which do not provide an inventive concept. The claims are not patent eligible.
Additionally, with regards to dependent claims 3, 7, 13 and 17, the limitation(s) of “wherein the pavement condition is determined by a machine learning model trained using… training data”, as drafted, are processes that, under their broadest reasonable interpretations, cover performance of the limitation(s) in the mind but for the recitation of generic computer components, a previously trained neural network. That is, other than reciting “a machine learning model trained using… training data”, nothing in the claim elements preclude the step(s) from practically being performed in the mind. The Examiner asserts that the claim(s) do not provide any details nor limit how the machine learning model operates or how the pavement condition is determined, and the plain meaning of determining encompasses mental observations, evaluations, judgments and/or opinions, e.g., a user observing imagery of a roadway and mentally deciding on the condition of the pavement of the roadway. Under its broadest reasonable interpretation when read in light of the specification, the determining encompasses mental observations, evaluations, judgments and/or opinions that are practically performed in the human mind. For example, but for the recitation of the aforementioned generic computer components, the determining the pavement condition encompasses a user observing imagery of a roadway and performing an evaluation, judgment and/or opinion to mentally decide on the condition of the pavement of the roadway in the imagery. If a claim limitation, under its broadest reasonable interpretation, covers performance of the limitation in the mind but for the recitation of generic computer components, then it falls within the “Mental Processes” grouping of abstract ideas. See MPEP § 2106.04(a)(2)(III). The claims are not patent eligible.
Furthermore, with regards to dependent claims 3, 7, 13 and 17, the limitation(s) of “a machine learning model” provide nothing more than mere instructions to implement an abstract idea on a generic computer. See MPEP § 2106.05(f). MPEP 2106.05(f) provides the following considerations for determining whether a claim simply recites a judicial exception with the words “apply it” (or an equivalent), such as mere instructions to implement an abstract idea on a computer: (1) whether the claim recites only the idea of a solution or outcome i.e., the claim fails to recite details of how a solution to a problem is accomplished; (2) whether the claim invokes computers or other machinery merely as a tool to perform an existing process; and (3) the particularity or generality of the application of the judicial exception. Moreover, the aforementioned machine learning model(s) are used to generally apply the abstract idea without placing any limits on how the aforementioned models function. See MPEP 2106.05(f). Additionally, the recitations of “a machine learning model” merely indicate a field of use or technological environment in which the judicial exception is performed. Although the additional elements of the aforementioned models limit the identified judicial exception of determining a pavement condition of a roadway, these types of limitations merely confine the use of the abstract idea to a particular technological environment (machine learning) and thus fail to add an inventive concept to the claims. See MPEP 2106.05(h). The claims are not patent eligible.
In addition, with regards to dependent claims 2 - 5, 7, 9, 10, 12 - 15, 17, 19 and 20, the Examiner asserts that claims 2 - 5, 7, 9, 10, 12 - 15, 17, 19 and 20 are also directed to the abstract idea of determining a pavement condition of a roadway and that dependent claims 2 - 5, 7, 9, 10, 12 - 15, 17, 19 and 20 merely further limit the abstract idea claimed in independent claims 1 and 11, for example by further identifying additional information used to determine the pavement condition, by further identifying general generic characteristics of the machine learning model(s) and/or by further identifying how the reflective value is used to determine the pavement condition. However, the Examiner asserts that a more detailed abstract idea remains an abstract idea and that none of the limitations of the dependent claims considered as an ordered combination provide eligibility because taken as a whole the claims merely instruct the practitioner to apply the abstract idea using generic computer components. The claims are not eligible.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 9 - 11, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette et al. U.S. Publication No. 2020/0226373 A1 in view of Jin et al. U.S. Publication No. 2022/0057325 A1.
- With regards to claims 1 and 11, Kottenstette et al. disclose a method, (Kottenstette et al., Abstract) and a system, (Kottenstette et al., Abstract, Figs. 26 - 28, Pg. 3 ¶ 0042 - 0043, Pg. 5 ¶ 0099 - Pg. 6 ¶ 0103, Pg. 6 ¶ 0106 - 0111, Pg. 7 ¶ 0119 - 0120, Pg. 24 ¶ 0298 and 0302 - 0304) comprising: a computing device comprising a processor and memory; (Kottenstette et al., Figs. 26 - 28, Pg. 3 ¶ 0042 - 0043, Pg. 5 ¶ 0099 - 0101, Pg. 6 ¶ 0103 - 0112, Pg. 7 ¶ 0118 - 0120) and an application for pavement condition assessment comprising machine readable instructions stored in the memory that, when executed by the processor, cause the computing device to (Kottenstette et al., Figs. 26 - 28, Pg. 3 ¶ 0042 - 0043, Pg. 5 ¶ 0099 - 0101, Pg. 6 ¶ 0103 - 0113, Pg. 7 ¶ 0117 - 0120) at least: obtain satellite imagery including a portion of a roadway; (Kottenstette et al., Figs. 10C, 13, 19A, 19B, 25 & 36, Pg. 5 ¶ 0092, 0095 and 0099, Pg. 5 ¶ 0101 - Pg. 6 ¶ 0102, Pg. 7 ¶ 0121, Pg. 19 ¶ 0241 - 0242, Pg. 21 ¶ 0263 - 0266, Pg. 22 ¶ 0286, Pg. 23 ¶ 0288 - 0291, Pg. 24 ¶ 0298 and 0304) determine a reflective value; (Kottenstette et al., Pg. 17 ¶ 0220 - 0221, Pg. 18 ¶ 0233 - 0234, Pg. 19 ¶ 0242, Pg. 21 ¶ 0262 - 0266, Pg. 23 ¶ 00288 - 0293 and 0296, Pg. 24 ¶ 0298 [“It can be useful and valuable to distinguish roof material attributes (e.g., wood vs tile shingles), as well as roof age at scale. Non-optimal data, such as low resolution satellite images, can be used. A given material can behave in a certain way with respect to reflectance, given known parameters (e.g., sun spectrum/angle/intensity, camera angles), and desired parameters (e.g., age, material). The desired parameters can be extracted from the image based on the hyperspectral data” and “systems and methods can be utilized to estimate conditions of a road, parcel, and/or roof (e.g., effective age)”]) determine a pavement condition of the portion of the roadway; (Kottenstette et al., Pg. 7 ¶ 0121, Pg. 19 ¶ 0241 - 0242, Pg. 22 ¶ 0279 - 0282, Pg. 23 ¶ 0289 - 0293 and 0296, Pg. 24 ¶ 0298 [“systems and methods can be utilized to estimate conditions of a road, parcel, and/or roof (e.g., effective age)”]) and identify a pavement management strategy based at least in part upon the pavement condition. (Kottenstette et al., Pg. 5 ¶ 0095 and 0101, Pg. 6 ¶ 0109 and 0113, Pg. 7 ¶ 0116 and 0121, Pg. 18 ¶ 0233 - 0235, Pg. 19 ¶ 0242, Pg. 22 ¶ 0279 - 0282, Pg. 23 ¶ 0288 - 0293, Pg. 24 ¶ 0298 [“analytical module 2706 can be configured to cause the processor 2702 or the imagery analysis server 2604 to extract, analyze, and/or label structural and/or geospatial information of any imagery data based on the techniques disclosed in the present disclosure” and “systems and methods can be utilized to estimate conditions of a road, parcel, and/or roof (e.g., effective age).”]) Kottenstette et al. fail to disclose expressly determining a reflective value corresponding to the roadway; and determining a pavement condition based at least in part upon the reflective value. Pertaining to analogous art, Jin et al. disclose obtaining imagery including a portion of a roadway; (Jin et al., Abstract, Figs. 6, 7 & 10 - 15, Pg. 1 ¶ 0008 and 0012, Pg. 2 ¶ 0044 and 0048 - 0050, Pg. 3 ¶ 0052, Pg. 4 ¶ 0065 - 0066 and 0068 - 0075) determining a reflective value corresponding to the portion of the roadway; (Jin et al., Abstract, Figs. 1 & 6, Pg. 1 ¶ 0008 - 0018, Pg. 2 ¶ 0044 - 0049, Pg. 4 ¶ 0064 - 0066) determining a pavement condition of the portion of the roadway based at least in part upon the reflective value; (Jin et al., Abstract, Fig. 6, Pg. 1 ¶ 0006 - 0008 and 0012 - 0016, Pg. 2 ¶ 0045 - 0050, Pg. 4 ¶ 0065 - 0066) and identifying a pavement management strategy based at least in part upon the pavement condition. (Jin et al., Pg. 1 ¶ 0003 and 0006 - 0008, Pg. 2 ¶ 0050, Pg. 4 ¶ 0064 - 0066, Pg. 5 ¶ 0093 - 0100) Kottenstette et al. and Jin et al. are combinable because they are both directed towards image processing systems that analyze reflectance measurements and estimate road conditions. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Kottenstette et al. with the teachings of Jin et al. This modification would have been prompted in order to enhance the base device of Kottenstette et al. with the well-known and applicable technique Jin et al. applied to a comparable device. Determining a pavement condition based at least in part upon a reflective value corresponding to a roadway, as taught by Jin et al., would enhance the base device of Kottenstette et al. by facilitating its ability to estimate conditions of roads in addition to conditions of roofs. In addition, this modification would enhance the base device of Kottenstette et al. by allowing for it to detect a wider variety of conditions of objects, such as weather-related conditions, and/or to be utilized in an increased number of related and applicable applications, such as for estimating road conditions, thereby improving its overall appeal, usefulness and marketability to potential end-users. Furthermore, this modification would have been prompted by the teachings and suggestions of Kottenstette et al. that reflectance can be used to extract desired parameters of an object, such as age and/or material, from an image and that their teachings can be used to estimate conditions of a road, such as its effective age, see at least page 23 paragraphs 029 - 0293 and 0296 and page 24 paragraph 0298 of Kottenstette et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the base device of Kottenstette et al. would determine a reflective value corresponding to the roadway so as to facilitate its ability to determine a pavement condition and estimate conditions of roads. Therefore, it would have been obvious to combine Kottenstette et al. with Jin et al. to obtain the invention as specified in claims 1 and 11.
- With regards to claims 9 and 19, Kottenstette et al. in view of Jin et al. disclose the method and the system of claims 1 and 11, respectively. Kottenstette et al. fail to disclose explicitly wherein the pavement condition of the portion of the roadway is determined based at least in part upon a plurality of reflective values, each reflective value associated with a band of the satellite imagery. Pertaining to analogous art, Jin et al. disclose wherein the pavement condition of the portion of the roadway is determined based at least in part upon a plurality of reflective values, each reflective value associated with a band of the satellite imagery. (Jin et al., Abstract, Fig. 6, Pg. 1 ¶ 0008 - 0016, Pg. 2 ¶ 0044 - 0049, Pg. 3 ¶ 0051 - 0057, Pg. 4 ¶ 0066)
- With regards to claims 10 and 20, Kottenstette et al. in view of Jin et al. disclose the method and the method of claims 1 and 11, respectively. Kottenstette et al. fail to disclose explicitly wherein the pavement condition of the portion of the roadway is determined based comparison of the reflective value to a defined threshold and variation of the reflective value over a period of time. Pertaining to analogous art, Jin et al. disclose wherein the pavement condition of the portion of the roadway is determined based comparison of the reflective value to a defined threshold (Jin et al., Abstract, Fig. 6, Pg. 1 ¶ 0008 and 0013 - 0016, Pg. 2 ¶ 0048 - 0049, Pg. 3 ¶ 0051 - 0060, Pg. 4 ¶ 0066, Pg. 5 ¶ 0090 - 0095) and variation of the reflective value over a period of time. (Jin et al., Pg. 3 ¶ 0051 - 0060, Pg. 4 ¶ 0064 - 0066, Pg. 5 ¶ 0093 - 0095)
Claims 2, 3, 5, 7, 12, 13, 15 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette et al. U.S. Publication No. 2020/0226373 A1 in view of Jin et al. U.S. Publication No. 2022/0057325 A1 as applied to claims 1 and 11 above, and further in view of Raufi et al. U.S. Publication No. 2023/0029746 A1.
- With regards to claims 2 and 12, Kottenstette et al. in view of Jin et al. disclose the method and the system of claims 1 and 11, respectively, wherein the pavement condition is determined based upon pavement surface conditions of the portion of the roadway. (Kottenstette et al., Pg. 7 ¶ 0121, Pg. 19 ¶ 0241 - 0242, Pg. 22 ¶ 0279 - 0282, Pg. 23 ¶ 0289 - 0293 and 0296, Pg. 24 ¶ 0298 [“systems and methods can be utilized to estimate conditions of a road, parcel, and/or roof (e.g., effective age).”]) Kottenstette et al. fail to disclose explicitly wherein the pavement condition is determined based upon pavement subsurface conditions. Pertaining to analogous art, Jin et al. disclose wherein the pavement condition is determined based upon pavement surface conditions of the portion of the roadway. (Jin et al., Abstract, Fig. 6, Pg. 1 ¶ 0006 - 0008 and 0012 - 0016, Pg. 2 ¶ 0045 - 0050, Pg. 4 ¶ 0065 - 0066) Jin et al. fail to disclose explicitly wherein the pavement condition is determined based upon pavement subsurface conditions. Pertaining to analogous art, Raufi et al. disclose wherein the pavement condition is determined based upon pavement subsurface conditions of the portion of the roadway. (Raufi et al., Pg. 1 ¶ 0007 - 0009, Pg. 2 ¶ 0020, Pg. 3 ¶ 0044 - 0046 and 0052 - 0055, Pg. 4 ¶ 0069 - 0072, Pg. 5 ¶ 0087, Pg. 6 ¶ 0091 - 0093) Kottenstette et al. in view of Jin et al. and Raufi et al. are combinable because they are all directed towards image processing systems that analyze imagery data and estimate road conditions and, similar to Kottenstette et al., Raufi et al. is also directed towards utilizing machine learning models to identify conditions of infrastructure objects. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Kottenstette et al. in view of Jin et al. with the teachings of Raufi et al. This modification would have been prompted in order to enhance the combined base device of Kottenstette et al. in view of Jin et al. with the well-known and applicable technique Raufi et al. applied to a similar device. Determining the pavement condition based upon pavement subsurface conditions, as taught by Raufi et al., would enhance the combined base device by allowing for it to detect a greater number and variety of conditions affecting the pavement, such as subsurface conditions that may not be visible by an above-ground observer, and thus enabling a more comprehensive view of the condition of the pavement to be obtained and improving its ability to estimate a reliable and accurate condition of the pavement. In addition, this modification would enhance the combined base device by allowing for it to be utilized in an increased number of related and applicable applications, such as for detecting subsurface damage to infrastructure, thereby improving its overall appeal, usefulness and marketability to potential end-users. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the combined base device would factor in pavement subsurface conditions when determining the pavement condition so as to enable a more comprehensive condition of the pavement to be obtained. Therefore, it would have been obvious to combine Kottenstette et al. in view of Jin et al. with Raufi et al. to obtain the invention as specified in claims 2 and 12.
- With regards to claims 3 and 13, Kottenstette et al. in view of Jin et al. in view of Raufi et al. disclose the method and the system of claims 2 and 12, respectively, wherein the pavement condition is determined by a machine learning model trained using measured training data. (Kottenstette et al., Abstract, Figs. 2, 4, 6, 7, 14, 15, 21 - 23, 28, 32A & 32B, Pg. 1 ¶ 0006 - 0007 and 0010 - 0013, Pg. 5 ¶ 0093 - 0097, Pg. 6 ¶ 0109, Pg. 7 ¶ 0114 - 0118, Pg. 9 ¶ 0134 - Pg. 10 ¶ 0140, Pg. 11 ¶ 0152 - 0155, Pg. 12 ¶ 0165 - Pg. 13 ¶ 0167, Pg. 17 ¶ 0218 - 0224 and 0227, Pg. 23 ¶ 0287 - 0293, Pg. 24 ¶ 0298 and 0303 - 0304) Kottenstette et al. fail to disclose explicitly a machine learning model trained using synthesized training data. Pertaining to analogous art, Raufi et al. disclose wherein the pavement condition is determined by a machine learning model trained using measured and synthesized training data. (Raufi et al., Abstract, Pg. 1 ¶ 0007 - 0009, Pg. 2 ¶ 0017, Pg. 4 ¶ 0069 - 0074, Pg. 5 ¶ 0082 - 0083) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Kottenstette et al. in view of Jin et al. in view of Raufi et al. with additional teachings of Raufi et al. This modification would have been prompted in order to enhance the combined base device of Kottenstette et al. in view of Jin et al. in view of Raufi et al. with well-known and applicable technique Raufi et al. applied to a similar device. Utilizing a machine learning model trained using synthesized training data, as taught by Raufi et al., would enhance the combined base device by improving its ability to accurately and reliably estimate a large variety of pavement conditions in a diversity of settings since synthesized training data, in addition to measured training data, would be used to train the machine learning model so as to ensure that the machine learning model is trained on a large and diverse dataset in order to enhance the robustness and generalization ability of the machine learning model. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the combined base device would utilize a machine learning model trained using synthesized, in addition to measured, training data so as to enhance the robustness and generalization ability of the machine learning model by ensuring that it is trained on a large and diverse training dataset and thereby improve its ability to accurately and reliably determine pavement conditions. Therefore, it would have been obvious to combine Kottenstette et al. in view of Jin et al. in view of Raufi et al. with additional teachings of Raufi et al. to obtain the invention as specified in claims 3 and 13.
- With regards to claims 5 and 15, Kottenstette et al. in view of Jin et al. in view of Raufi et al. disclose the method and the system of claims 3 and 13, respectively. Kottenstette et al. fail to disclose explicitly wherein the synthesized training data is generated based upon the measured training data. Pertaining to analogous art, Raufi et al. disclose wherein the synthesized training data is generated based upon the measured training data. (Raufi et al., Pg. 1 ¶ 0009, Pg. 2 ¶ 0017, Pg. 4 ¶ 0069 - 0075, Pg. 5 ¶ 0082 - 0083)
- With regards to claims 7 and 17, Kottenstette et al. in view of Jin et al. in view of Raufi et al. disclose the method and the system of claims 2 and 12, respectively, wherein the pavement condition is determined by a machine learning model trained using training data. (Kottenstette et al., Abstract, Figs. 6, 14, 15, 21 - 23, 28, 32A & 32B, Pg. 1 ¶ 0006 - 0007 and 0010 - 0013, Pg. 5 ¶ 0093 - 0097, Pg. 6 ¶ 0109, Pg. 7 ¶ 0114 - 0118, Pg. 9 ¶ 0134 - Pg. 10 ¶ 0140, Pg. 11 ¶ 0152 - 0155, Pg. 12 ¶ 0165 - Pg. 13 ¶ 0167, Pg. 17 ¶ 0218 - 0224 and 0227, Pg. 23 ¶ 0291, Pg. 24 ¶ 0298 and 0303 - 0304) Kottenstette et al. fail to disclose explicitly using synthesized and feature generated training data. Pertaining to analogous art, Raufi et al. disclose wherein the pavement condition is determined by a machine learning model trained using synthesized and feature generated training data. (Raufi et al., Abstract, Pg. 1 ¶ 0007 - 0009, Pg. 2 ¶ 0017, Pg. 4 ¶ 0069 - 0075, Pg. 5 ¶ 0082 - 0084) It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Kottenstette et al. in view of Jin et al. in view of Raufi et al. with additional teachings of Raufi et al. This modification would have been prompted in order to enhance the combined base device of Kottenstette et al. in view of Jin et al. in view of Raufi et al. with well-known and applicable technique Raufi et al. applied to a similar device. Utilizing a machine learning model trained using synthesized and feature generated training data, as taught by Raufi et al., would enhance the combined base device by improving its ability to accurately and reliably estimate a large variety of pavement conditions in a diversity of settings since synthesized and feature generated training data, in addition to measured training data, would be used to train the machine learning model so as to ensure that the machine learning model is trained on a large and diverse dataset in order to enhance the robustness and generalization ability of the machine learning model. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the combined base device would utilize a machine learning model trained using synthesized and feature generated training data, in addition to measured training data, so as to enhance the robustness and generalization ability of the machine learning model by ensuring that it is trained on a large and diverse training dataset and thereby improve its ability to accurately and reliably determine pavement conditions. Therefore, it would have been obvious to combine Kottenstette et al. in view of Jin et al. in view of Raufi et al. with additional teachings of Raufi et al. to obtain the invention as specified in claims 7 and 17.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette et al. U.S. Publication No. 2020/0226373 A1 in view of Jin et al. U.S. Publication No. 2022/0057325 A1 in view of Raufi et al. U.S. Publication No. 2023/0029746 A1 as applied to claims 3 and 13 above, and further in view of Szuba et al. U.S. Publication No. 2023/0360402 A1.
- With regards to claims 4 and 14, Kottenstette et al. in view of Jin et al. in view of Raufi et al. disclose the method and the system of claims 3 and 13, respectively, wherein the machine learning model is a machine learning model. (Kottenstette et al., Pg. 1 ¶ 0011, Pg. 5 ¶ 0093 - 0097, Pg. 9 ¶ 0134, Pg. 10 ¶ 0140, Pg. 11 ¶ 0155, Pg. 22 ¶ 0286 - Pg. 23 ¶ 0291, Pg. 24 ¶ 0298 and 0303) Kottenstette et al. fail to disclose explicitly a YOLOR machine learning model. Pertaining to analogous art, Szuba et al. disclose wherein the machine learning model is a YOLOR machine learning model. (Szuba et al., Fig. 6, Pg. 4 ¶ 0048, Pg. 6 ¶ 0070 - 0073, Pg. 7 ¶ 0075 - 0077) Kottenstette et al. in view of Jin et al. in view of Raufi et al. and Szuba et al. are combinable because they are all directed towards image processing systems that analyze imagery data and, similar to Kottenstette et al. and Raufi et al., Szuba et al. is also directed towards utilizing machine learning models to detect and determine characteristics or properties of objects in images. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Kottenstette et al. in view of Jin et al. in view of Raufi et al. with the teachings of Szuba et al. This modification would have been prompted in order to substitute the machine learning model of Kottenstette et al. for the YOLOR machine learning model of Szuba et al. The YOLOR machine learning model of Szuba et al. could be substituted in place of the machine learning model of Kottenstette et al. using well-known techniques in the art and would likely yield predictable results, in that in the combination a YOLOR machine learning model would be trained and utilized to determine pavement conditions. Furthermore, this modification would have been prompted by the teachings and suggestions of Kottenstette et al. that deep learning frameworks and convolutional neural networks can be used to classify data, and that their teachings can be applied to any type of machine learning or deep learning architecture, see at least page 5 paragraphs 0093 - 0094, page 9 paragraph 0134, page 10 paragraph 0140 and page 24 paragraphs 0298 and 0303 of Kottenstette et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the combined base device would utilize a YOLOR machine learning model to determine pavement conditions. Therefore, it would have been obvious to combine Kottenstette et al. in view of Jin et al. in view of Raufi et al. with Szuba et al. to obtain the invention as specified in claims 4 and 14.
Claims 6, 8, 16 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kottenstette et al. U.S. Publication No. 2020/0226373 A1 in view of Jin et al. U.S. Publication No. 2022/0057325 A1 in view of Raufi et al. U.S. Publication No. 2023/0029746 A1 as applied to claims 5, 7, 15 and 17 above, and further in view of Gonzalez Diaz et al. U.S. Publication No. 2023/0153982 A1.
- With regards to claims 6 and 16, Kottenstette et al. in view of Jin et al. in view of Raufi et al. disclose the method and the method of claims 5 and 15, respectively. Kottenstette et al. fail to disclose explicitly wherein the synthesized training data is generated from incorporation of a fault in measured training data. Pertaining to analogous art, Gonzalez Diaz et al. disclose wherein the synthesized training data is generated from incorporation of a fault in measured training data. (Gonzalez Diaz et al., Abstract, Figs. 1, 2, 6, 8, 11 & 12, Pg. 1 ¶ 0002 - 0003, Pg. 1 ¶ 0011 - Pg. 2 ¶ 0012, Pg. 3 ¶ 0035 - 0039, Pg. 4 ¶ 0047 - 0048 and 0053, Pg. 5 ¶ 0065 - 0067, Pg. 6 ¶ 0079) Kottenstette et al. in view of Jin et al. in view of Raufi et al. and Gonzalez Diaz et al. are combinable because they are all directed towards image processing systems that analyze imagery data and, similar to Kottenstette et al. and Raufi et al., Gonzalez Diaz et al. is also directed towards training machine learning models based on imagery data. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Kottenstette et al. in view of Jin et al. in view of Raufi et al. with the teachings of Gonzalez Diaz et al. This modification would have been prompted in order to substitute the synthetic training data creation process of Raufi et al. for the synthesized training data generation technique of Gonzalez Diaz et al. The synthesized training data generation technique of Gonzalez Diaz et al. could be substituted in place of the synthetic training data creation process of Raufi et al. using well-known techniques in the art and would likely yield predictable results, in that in the combination the synthesized training data generation technique of Gonzalez Diaz et al. that incorporates damage, i.e., faults, into measured data would be utilized to generate the synthesized training data. Furthermore, this modification would have been prompted by the teachings and suggestions of Raufi et al. that they create synthetic labeled training data, that they simulate discrepancy of segments have to be merged in real cases, and that they are interested in detected damage under or around infrastructure, such as roads, see at least page 5 paragraphs 0082 - 0084 and page 6 paragraphs 0091 - 0093 of Raufi et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the synthesized training data generation technique of Gonzalez Diaz et al. that incorporates damage, i.e., faults, into measured data would be utilized to generate the synthesized training data of the combined base device. Therefore, it would have been obvious to combine Kottenstette et al. in view of Jin et al. in view of Raufi et al. with Gonzalez Diaz et al. to obtain the invention as specified in claims 6 and 16.
- With regards to claims 8 and 18, Kottenstette et al. in view of Jin et al. in view of Raufi et al. disclose the method and the system of claims 7 and 17, respectively. Kottenstette et al. fail to disclose explicitly wherein the feature generated training data is generated using variational auto-encoder (VAE) based augmentation with shortcut connections. Pertaining to analogous art, Gonzalez Diaz et al. disclose wherein the feature generated training data is generated using variational auto-encoder (VAE) based augmentation with shortcut connections. (Gonzalez Diaz et al., Abstract, Figs. 1, 2, 6, 8, 11 & 12, Pg. 1 ¶ 0002 - 0003, Pg. 1 ¶ 0011 - Pg. 2 ¶ 0012, Pg. 3 ¶ 0035 - 0039, Pg. 4 ¶ 0047 - 0048 and 0053, Pg. 5 ¶ 0065 - 0067, Pg. 6 ¶ 0079 [“In an example implementation of a generator, there is an encoder-decoder component that captures features from the images at multiple levels with skip connections”.]) Kottenstette et al. in view of Jin et al. in view of Raufi et al. and Gonzalez Diaz et al. are combinable because they are all directed towards image processing systems that analyze imagery data and, similar to Kottenstette et al. and Raufi et al., Gonzalez Diaz et al. is also directed towards training machine learning models based on imagery data. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the combined teachings of Kottenstette et al. in view of Jin et al. in view of Raufi et al. with the teachings of Gonzalez Diaz et al. This modification would have been prompted in order to substitute the feature generated training data creation process of Raufi et al. for the feature generated training data generation technique of Gonzalez Diaz et al. The feature generated training data generation technique of Gonzalez Diaz et al. could be substituted in place of the feature generated training data creation process of Raufi et al. using well-known techniques in the art and would likely yield predictable results, in that in the combination the feature generated training data generation technique of Gonzalez Diaz et al. that uses variational auto-encoder (VAE) based augmentation with shortcut connections would be utilized to generate the feature generated training data. Furthermore, this modification would have been prompted by the teachings and suggestions of Kottenstette et al. that deep learning frameworks and autoencoders can be used for pattern analysis or synthesis, and that their teachings can be applied to any type of machine learning or deep learning architecture, see at least page 5 paragraphs 0093 - 0094, page 22 paragraph 0286 - page 23 paragraph 0287 and page 24 paragraphs 0298 and 0303 of Kottenstette et al. Moreover, this modification would have been prompted by the teachings and suggestions of Raufi et al. that they create synthetic labeled training data, that they simulate discrepancy of segments have to be merged in real cases, and that they are interested in detected damage under or around infrastructure, such as roads, see at least page 5 paragraphs 0082 - 0084 and page 6 paragraphs 0091 - 0093 of Raufi et al. This combination could be completed according to well-known techniques in the art and would likely yield predictable results, in that the feature generated training data generation technique of Gonzalez Diaz et al. that uses variational auto-encoder (VAE) based augmentation with shortcut connections would be utilized to generate the feature generated training data of the combined base device. Therefore, it would have been obvious to combine Kottenstette et al. in view of Jin et al. in view of Raufi et al. with Gonzalez Diaz et al. to obtain the invention as specified in claims 8 and 18.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Fukamizu et al. U.S. Patent No. 4,690,553; which is directed towards a method and system for detecting a condition of a road surface, wherein reflectance values are evaluated to detect the condition of the road surface.
Nien et al. U.S. Publication No. 2025/0366818 A1; which is directed towards a method and apparatus for automatically detecting and classifying an object of interest in image data, wherein a You Only Learn One Representation (YOLOR) machine learning model is used to detect and classify the object of interest.
Ramos et al. U.S. Publication No. 2024/0167962 A1; which is directed towards a system and method for monitoring pavement condition, wherein image data is analyzed by a machine learning model to identify the presence and/or location of pavement distresses on a pavement surface.
Ruzanski et al. U.S. Publication No. 2019/0180458 A1; which is directed towards systems and methods for estimating road conditions, wherein a machine learning model processes satellite images to estimate a pavement state of pavement.
Sherman et al. U.S. Publication No. 2022/0415051 A1; which is directed towards methods and systems for identifying objects in images, wherein a You Only Learn One Representation (YOLOR) machine learning model is used to determine the location of objects in images.
Zhang et al. U.S. Publication No. 2022/0276182 A1; which is directed towards a system and method recognizing damage of asphalt pavement, wherein a machine learning model processes image data to recognize moisture damage of the asphalt pavement.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC RUSH whose telephone number is (571) 270-3017. The examiner can normally be reached 9am - 5pm Monday - Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270 - 5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ERIC RUSH/Primary Examiner, Art Unit 2677
Prosecution Insights