Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1 and 3-5 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Chisato (WO2023042238).
With respect to claim 1, Chisato teaches a method of determining and displaying pavement conditions on a road surface. It is understood that the road surface could be that of an airport. Chisato teaches determining an orthomosaic of the airport pavement (via a road sensor) for scanning the pavement with a unmanned aerial systems (drone or other mobile unit – see page 3/13 paras. 8 and 9). Chisato teaches generating a plurality of subsections (road coordinates/section areas, see page 2 last paragraph and page 3/13, last paragraph). Chisato teaches a generator unit 103 for analyzing one or more subsections to determine a pavement condition by assessing an index factor pursuant to (Maintenance Control Index (MCI) or International Road Index (IRI) – see page 3/13, para. 2). Chisato further teaches using deep neural networks to classify the subsections of the pavement (see page 7/13, 6th full para. and page 6/13, 6th full paragraph. For example, Chisato teaches: “The generation unit 103 may generate a predicted image by inputting the road image into a learning model generated by a deep learning model. As a learning model, for example, a Generative Adversarial Network (GAN) may be used. For example, among GANs, Cycle-GAN or Pix2Pix may be used. The aforementioned stored images may be used to generate the learning model.”
Chisato teaches displaying (page 7/13, 9th para. – page 8/13, 5th para). The image generation system 100 may further include a display control unit that displays the predicted image on a display (not shown). The display is, for example, a display connected to a computer or a tablet.
With respect to claim 3, Chisato teaches a generator unit 103 or analyzing one or more subsections to determine a pavement condition by assessing an index factor pursuant to (Maintenance Control Index (MCI) or International Road Index (IRI) – see page 3/13, para. 2).
With respect to claim 4, Chisato further teaches using neural networks to classify the subsections of the pavement (see page 7/13, 6th full para. and page 6/13, 6th full paragraph. Chisato teaches displaying (page 7/13, 9th para. – page 8/13, 5th para).
With respect to claim 5, Chisato teaches using a deep learning model using neural networks to classify the subsections of the pavement (see page 7/13, 6th full para. and page 6/13, 6th full paragraph. Chisato teaches displaying (page 7/13, 9th para. – page 8/13, 5th para). Chisato teaches generating a plurality of subsections (road coordinates/section areas, see page 2 last paragraph and page 3/13, last paragraph). Chisato teaches a generator unit 103 or analyzing one or more subsections to determine a pavement condition by assessing and index factor pursuant to (Maintenance Control Index (MCI) or International Road Index (IRI) – see page 3/13, para. 2).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 2, 6 – 8 and 10-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chisato (WO2023042238) in view of KSR v. Teleflex 550 U.S. 398 (2007).
With respect to claim 2, Chisato teaches all of the subject matter upon which the claim is based except for the presence of at least one pavement condition under the ASTM D5340 standard.
Chisato clearly teaches analyzing a pavement under the MCI/ICI standard as set forth in the rejection to claim 1.
The examiner contends that one or more conditions under ASTM D5340 standard are commensurate with one or more standards under MCI/ICI standards as set forth by Chisato. For example, various indicators are used to indicate the degree of road deterioration. In the present disclosure, the degree of road deterioration is represented by the degree of deterioration. The degree of deterioration may be any index including degree of cracking, number of potholes, size of potholes, amount of rutting, or flatness. Also, the degree of deterioration may be determined based on a combination of a plurality of indicators representing the degree of road deterioration.
The degree of cracking is represented by the shape, length, area, number of cracks, or a combination thereof. Crack rate is an example of crack degree. The crack ratio is represented by, for example, 100×(crack area/road section area). In this case, the value of the degree of deterioration ranges from 0% to 100.
Hence, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to try different or otherwise known standards suitable for analyzing the pavement. Furthermore, one of ordinary skill in the art would have known to substitute the ASTM D5340 standard or any other known standards suitable for classifying the aberrations associated with the pavement of an airport.
With respect to claim 6, Chisato teaches a method wherein a plurality of images of a pavement (page 4/13, 2nd para.). Chisato teaches using a computer (generation unit 103) for combining and processing the plurality of images.
What Chisato does not specifically show is the overlap in sections while the image is captured.
The Examiner contends that an overlap of image scanning is contemplated or would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, in that generation unit 103 would take images and process them. Such image frames would cover sections of the pavement that have been covered in previous frames or by previous swaths from a previous time in which the road had been scanned. The scanning overlap appears to be required to assure that no area of the pavement had been omitted in its view of deteriorated surfaces.
With respect to claim 7, Chisato teaches scanning a payment using a plurality of regions. In the third paragraph of the Embodiment, Chisato teaches an acquisition unit 101 which acquires a road image.. The road image may be captured by an in-vehicle camera such as a drive recorder. However, the type of camera is not limited to this, and various types of cameras may be used. For example, the road image may be captured by a camera mounted on a bicycle, a drone, or other mobile object, a camera carried by a person, or a fixed camera installed on the road. The road image may be taken by a person or automatically.
An image generation system according to the present disclosure includes acquisition means for acquiring a road image obtained by imaging a road, deterioration level determination means for determining a future deterioration level of the road, and based on the road image, according to the deterioration level, and generating means for generating a prediction image representing the road deterioration on the road.
What is not taught by Chisato is the specific pixel resolution of .29 cm to 1 cm.
However, the Examiner contends that would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the acquisition unit 101, so that the cameras associated with the unit, gather data at various resolutions, including those set forth by the claim.
With respect to claim 8, Chisato teaches using aerial drones with cameras located thereon for taking aerial photos of pavement for predicting deterioration as well as discovering it. While the specifics of the altitude, pitch and yaw of the gimbal are not specified by Chisato, the Examiner contends that it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to try variations of drone data gathering, for example, at specifics heights above the payment at such miles per hour and to snap images at certain rates per sec at particular angles to optimize the detection of deterioration in the pavement of structures.
With respect to claim 10, Chisato further teaches using neural networks to classify the subsections of the pavement (see page 7/13, 6th full para. and page 6/13, 6th full paragraph. Chisato teaches displaying (page 7/13, 9th para. – page 8/13, 5th para).
Chisato teaches all of the subject matter upon which the claim is based except for the presence of at least one pavement condition under ASTM D5340 standard.
The examiner contends that one or more conditions under ASTM D5340 standard are commensurate with one or more standards under MCI/ICI standards as set forth by Chisato. For example, various indicators are used to indicate the degree of road deterioration. In the present disclosure, the degree of road deterioration is represented by the degree of deterioration. The degree of deterioration may be any index including degree of cracking, number of potholes, size of potholes, amount of rutting, or flatness. Also, the degree of deterioration may be determined based on a combination of a plurality of indicators representing the degree of road deterioration. The degree of cracking is represented by the shape, length, area, number of cracks, or a combination thereof. Crack rate is an example of crack degree. The crack ratio is represented by, for example, 100×(crack area/road section area). In this case, the value of the degree of deterioration ranges from 0% to 100.
The Examiner contends that one or more conditions under ASTM D5340 standard are also covered under one or more of the MCI/ICI standards as set forth by Chisato. Furthermore, the examiner contends that it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to try a reasonable number of standards for detecting the condition of road pavements as clearly suggested by Chisato.
With respect to claim 11, Chisato teaches combining a visual image (taken by acquisition unit 101) of the road surface with representations of pavement stress (see page 2 paras. 8-10) at corresponding location coordinates.
With respect to claim 12, Chisato teaches a unit 103 for determining pavement conditions; determining a forecast pavement condition (deterioration degree determination unit 102), see page 4, paragraphs 3-9 and page 5 paragraphs 2-6 based on the determined pavement condition and forecast of maintenance scheduling and presenting the forecast to the user (page 7, para. 9 thru page 8, para 5).
With respect to claim 13, Chisato teaches an image generation system/apparatus 100 for determining and displaying pavement conditions. It is understood that the road surface could be that of an airport. Chisato teaches determining an orthomosaic of the airport pavement (via a road sensor) for scanning the pavement with a unmanned aerial systems (drone or other mobile unit – see page 3/13 paras. 8 and 9). Chisato teaches generating a plurality of subsections (road coordinates/section areas, see page 2 last paragraph and page 3/13, last paragraph). Chisato teaches a generator unit 103 for analyzing one or more subsections to determine a pavement condition by assessing and index factor pursuant to (Maintenance Control Index (MCI) or International Road Index (IRI) – see page 3/13, para. 2). Chisato further teaches using a deep network model to classify the subsections of the pavement (see page 7/13, 6th full para. and page 6/13, 6th full paragraph.
Chisato teaches a displaying system (page 7/13, 9th para. – page 8/13, 5th para). The image generation system 100 may further include a display control unit that displays the predicted image on a display (not shown). The display is, for example, a display connected to a computer or a tablet.
Chisato teaches all of the subject matter upon which the claim is based except for the presence of at least one pavement condition under ASTM D5340 standard.
The Examiner contends that one or more conditions under ASTM D5340 standard are also covered under one or more of the MCI/ICI standards as set forth by Chisato. For example, various indicators are used to indicate the degree of road deterioration. In the present disclosure, the degree of road deterioration is represented by the degree of deterioration. The degree of deterioration may be any index including degree of cracking, number of potholes, size of potholes, amount of rutting, or flatness. Also, the degree of deterioration may be determined based on a combination of a plurality of indicators representing the degree of road deterioration.
The Examiner contends that it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to try a reasonable number of standards for detecting the condition of road pavements as clearly suggested by Chisato.
With respect to claim 14, Chisato teaches a unit 103 for determining pavement conditions; determining a forecast pavement condition (deterioration degree determination unit 102), see page 4, paragraphs 3-9 and page 5 paragraphs 2-6. Chisato teaches that after determining the conditions of the pavement, a forecast of maintenance schedule may be conducted. Furthermore, forecast information may be presented to the user (page 7, para. 9 thru page 8, para 5). The motivation for the rejection is the same as that to claim 13.
With respect to claim 15, Chisato teaches wherein the system is configured to determine a predicted pavement condition from the determined pavement condition and a forecast maintenance schedule (page 5, para. 6) and forecast usage schedule (due to road deterioration, see page 4/13 at para. 2 and page 6/13 at para. 3). The motivation for the rejection of this claim is the same as that to claim 13.
Claim(s) 9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Chisato in view of KSR v. Teleflex 550 U.S. 398 (2007) further in view of KR101847944.
Chisato in view of KSR v. Teleflex teaches all of the subject matter upon which the claim depends except for more than one source of images taken from the imager.
The 944 reference teaches the use of multiple sources of imaging (see page 5/10 of para. 4).
Since Chisato and the 944 reference are both directed toward determining the condition of road pavements using aerial devices, the purpose of using multiple sources of imaging, would have been recognized by Chisato as set forth by the 944 reference.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify the image acquisition unit 101 to include multiple imaging sources on the drone, as taught by the 944 reference for the purpose of enhancing (taking multiple perspectives at different resolutions) for determination of defects in pavements.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEROME GRANT II whose telephone number is (571)272-7463. The examiner can normally be reached M-F 9:00 a.m. - 5:00 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEROME GRANT II/Primary Examiner, Art Unit 2664