DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Status
Claims 1-20 are pending.
Response to Arguments
Applicant's arguments filed 02/17/2026 regarding claim 1-16 have been fully considered but they are not persuasive. Applicant argues the previous prior art references of record fail to teach “retrieving, by the virtual audit application, from the geographic image database comprising pre-existing street view images captured along roads”. Examiner disagrees because Fig. 5 of Perkins shows a “pre-existing” street view image.
Applicant’s arguments, see Remarks page 8, filed 02/17/2026, with respect to claims 17-20 have been fully considered and are persuasive. The 103 rejection of claims 17-20 has been withdrawn.
In view of applicant’s amendments, a new grounds of rejection are made in view of newly cited references Cohen and Jayawickrema.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 13-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The following is Examiner's analysis of the claimed invention under the 2019 Revised Patent Subject Matter Eligibility Guidance (PEG)
STEP 1 ls the claim to a Process, Machine, Manufacture or Composition of matter? Yes. Claim 13 recites a process (method), claim 17 recites a process (method).
STEP2A Prong one: Does The Claim Recite An Abstract Idea, Law Of Nature, or Natural Phenomenon? Yes. Claim 13 recites “analyzing the plurality of images to identify and classify assets associated with the equipment” which falls within the mental processes grouping of abstract ideas. The step of “analyzing images” covers performance of the limitation in the mind; therefore, the claim recites an abstract idea.
Claim 13 recites “fails to identify one or more other assets associated with the equipment based on rules encoding physical coexistence relationships between asset types, wherein the rules specify that detection of a first asset type requires presence of a second asset type associated with characteristics of the equipment” which falls within the mental processes grouping of abstract ideas. The step of “failing to identifying assets” covers performance of the limitation in the mind; therefore, the claim recites an abstract idea.
Claim 17 recites “determining a route in the geographic region along which infrastructure equipment is installed” which falls within the mental processes grouping of abstract ideas. The step of “determining a route” covers performance of the limitation in the mind; therefore, the claim recites an abstract idea.
Claim 17 recites “comparing the first image acquisition timestamp information from the previous audit record to second image acquisition timestamp information associated with a plurality of second images of assets associated with the equipment identified at the plurality of locations in the previous audit record to identify which street view images have been updated since the previous audit” which falls within the mental processes grouping of abstract ideas. The step of comparing timestamp information covers performance of the limitation in the mind; therefore, the claim recites an abstract idea.
Claim 17 recites “determining based on the comparing, that at least one of the plurality of first images associated with a first location of the plurality of locations is acquired more recently than a respective one of the plurality of second images associated with the same first location” which falls within the mental processes grouping of abstract ideas. The step of determining recency covers performance of the limitation in the mind; therefore, the claim recites an abstract idea.
Claim 17 recites “analyzing only the at least one of the plurality of first images that is acquired more recently to identify and classify assets associated with the equipment” which falls within the mental processes grouping of abstract ideas. The step of “analyzing images to identify and classify assets” covers performance of the limitation in the mind; therefore, the claim recites an abstract idea.
STEP2A Prong two: Does The Claim Recite Additional Elements That Integrate The Judicial Exception Into A Practical Application? No. Claim 13 recites “a computer-implemented method of evaluating and updating a machine learning (ML) model for virtual auditing of infrastructure equipment along routes based on rules associated with physical installation characteristics of the infrastructure equipment, the method comprising” which amounts to merely including instructions to implement an abstract idea on a computer.
Claim 13 recites “receiving, by a virtual audit application stored in non-transitory memory of a computer system and executable by a processor of the computer system, a type of equipment to be audited and a geographic region; retrieving, by the virtual audit application, a plurality of images of the geographic region from a geographic image database comprising pre-existing street view images captured along roads” which is mere necessary data gathering.
Claim 13 recites “by the virtual audit application, using one or more ML models trained for autonomous infrastructure asset identification; wherein the analyzing comprises: processing a first image of the plurality of images using a first ML model of the one or more ML models to identify a first asset of the assets; determining, by an ML model training application stored in the non-transitory memory of the computer system and executable by the processor of the computer system, that the first ML model fails to” which amounts to merely including instructions to implement an abstract idea on a computer.
Claim 13 recites “updating, by the ML model training application, based on the determining, one or more parameters of the first ML model” which is insignificant-extra solution activity tangentially related to the invention.
Claim 17 recites “receiving, by a virtual audit application stored in non-transitory memory of a computer system and executable by a processor of the computer system, a type of equipment to be audited, a geographic region, and a previous audit record of the type of equipment in the geographic region, wherein the previous audit record includes timestamp information of images previously analyzed” which is mere necessary data gathering.
Claim 17 recites “by the virtual audit application” which amounts to merely including instructions to implement an abstract idea on a computer.
Claim 17 recites “retrieving, by the virtual audit application, from a geographic image database comprising street view images, first image acquisition timestamp information associated with a plurality of first images corresponding respectively to a plurality of locations along the route” which is mere necessary data gathering.
Claim 17 recites “by the virtual audit application, using one or more machine learning (ML) models trained for infrastructure equipment identification” which amounts to merely including instructions to implement an abstract idea on a computer.
Claim 17 recites “updating, by the virtual audit application, the previous audit record based on information about the assets identified from only the at least one of the plurality of first images that is acquired more recently” which is insignificant-extra solution activity tangentially related to the invention. Adding a final step of updating a record does not add a meaningful limitation to the judicial exception, and therefore, the additional element is insignificant-extra solution activity.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No. The courts have determined merely including instructions to implement the abstract idea on a computer does not qualify as “significantly more” when recited in a claim with a judicial exception (See Alice Corp., 573 U.S. at 225-26, 110 USPQ2d at 1984).
The courts have determined mere data gathering to not be enough to qualify as “significantly more” when recited in a claim with a judicial exception (See CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011)).
Claim 13 recites “updating, by the ML model training application, based on the determining, one or more parameters of the first ML model” which is storing information in memory. The courts have determined storing and retrieving information in memory is well-understood, routine, and conventional functionality when claimed in a merely generic manner (see Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)).
Claim 17 recites “updating, by the virtual audit application, the previous audit record based on information about the assets identified from only the at least one of the plurality of first images that is acquired more recently” which is storing information in memory. The courts have determined storing and retrieving information in memory is well-understood, routine, and conventional functionality when claimed in a merely generic manner (see Versata Dev. Group, Inc. v. SAP Am., Inc., 793 F.3d 1306, 1334, 115 USPQ2d 1681, 1701 (Fed. Cir. 2015)).
There is no indication that the elements of the claim, individually nor in combination, integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
For the reasons above, claims 13 and 17 are rejected as being directed to nonpatentable subject
matter under §101. This rejection applies equally to the dependent claims. The additional limitations
of the dependent claims are addressed briefly below:
Regarding claim 14
STEP 1 ls the claim to a Process, Machine, Manufacture or Composition of matter? Yes. The claim recites a process (method).
STEP2A Prong one: Does The Claim Recite An Abstract Idea, Law Of Nature, or Natural Phenomenon? Yes. The claim recites “further identify the one or more other assets determined based on the rules associated with the characteristics of the equipment” which falls within the mental processes grouping of abstract ideas.
STEP2A Prong two: Does The Claim Recite Additional Elements That Integrate The Judicial Exception Into A Practical Application? No. The claim recites “wherein the updating the one or more parameters of the first ML model comprises: training, by the ML model training application, the first ML model to” which amounts to merely including instructions to implement an abstract idea on a computer.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No. The courts have determined mere data gathering to not be enough to qualify as “significantly more” when recited in a claim with a judicial exception (See CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011)).
Regarding claim 15
STEP 1 ls the claim to a Process, Machine, Manufacture or Composition of matter? Yes. The claim recites a process (method).
STEP2A Prong one: Does The Claim Recite An Abstract Idea, Law Of Nature, or Natural Phenomenon? Yes. The claim inherits the abstract idea of the parent claim.
STEP2A Prong two: Does The Claim Recite Additional Elements That Integrate The Judicial Exception Into A Practical Application? No. The claim recites “wherein the rules associated with the characteristics of the equipment comprises a comparison against an external data source having records of assets associated with the equipment” which is mere necessary data gathering.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No. The courts have determined mere data gathering to not be enough to qualify as “significantly more” when recited in a claim with a judicial exception (See CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011)).
Regarding claim 16
STEP 1 ls the claim to a Process, Machine, Manufacture or Composition of matter? Yes. The claim recites a process (method).
STEP2A Prong one: Does The Claim Recite An Abstract Idea, Law Of Nature, or Natural Phenomenon? Yes. The claim recites “wherein the one or more other assets determined based on the rules comprise at least one of a power supply, a splice enclosure, telephony equipment, a tap, an amplifier, or a transformer” which is additional information about the abstract idea of the parent claim.
STEP2A Prong two: Does The Claim Recite Additional Elements That Integrate The Judicial Exception Into A Practical Application? No. There is no indication that the elements of the claim
integrate the judicial exception into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No. There is no indication that the elements of the claim, individually nor in
combination, integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Regarding claim 18
STEP 1 ls the claim to a Process, Machine, Manufacture or Composition of matter? Yes. The claim recites a process (method).
STEP2A Prong one: Does The Claim Recite An Abstract Idea, Law Of Nature, or Natural Phenomenon? Yes. The claim inherits the abstract idea of the parent claim.
STEP2A Prong two: Does The Claim Recite Additional Elements That Integrate The Judicial Exception Into A Practical Application? No. The claim recites “wherein the retrieving comprises: searching, by the virtual audit application, the geographic image database, for each location of the plurality of locations, for a timestamp associated with a most recent image that is closest to the respective location” which is mere necessary data gathering.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No. The courts have determined mere data gathering to not be enough to qualify as “significantly more” when recited in a claim with a judicial exception (See CyberSource v. Retail Decisions, Inc., 654 F.3d 1366, 1375, 99 USPQ2d 1690, 1694 (Fed. Cir. 2011)).
Regarding claim 19
STEP 1 ls the claim to a Process, Machine, Manufacture or Composition of matter? Yes. The claim recites a process (method).
STEP2A Prong one: Does The Claim Recite An Abstract Idea, Law Of Nature, or Natural Phenomenon? Yes. The claim recites “determining, by the virtual audit application, that the route corresponds to a route used for auditing the type of equipment in the previous audit record” which falls within the mental processes grouping of abstract ideas.
STEP2A Prong two: Does The Claim Recite Additional Elements That Integrate The Judicial Exception Into A Practical Application? No. There is no indication that the elements of the claim
integrate the judicial exception into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No. There is no indication that the elements of the claim, individually nor in
combination, integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Regarding claim 20
STEP 1 ls the claim to a Process, Machine, Manufacture or Composition of matter? Yes. The claim recites a process (method).
STEP2A Prong one: Does The Claim Recite An Abstract Idea, Law Of Nature, or Natural Phenomenon? Yes. The claim recites “wherein the identified assets comprise at least one of a power supply, a splice enclosure, telephony equipment, a tap, an amplifier, or a transformer” which is additional information about the abstract idea of the parent claim.
STEP2A Prong two: Does The Claim Recite Additional Elements That Integrate The Judicial Exception Into A Practical Application? No. There is no indication that the elements of the claim
integrate the judicial exception into a practical application.
Step 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? No. There is no indication that the elements of the claim, individually nor in
combination, integrate the judicial exception into a practical application or amount to significantly more than the judicial exception.
Taken alone, the additional elements of the dependent claims do not amount to significantly
more than the above-identified judicial exception (the abstract idea). Looking at the limitations as an
ordered combination adds nothing that is not already present when looking at the elements taken
individually. There is no indication that the combination of elements improves the functioning of a
computer or improves any other technology. Their collective functions merely provide conventional
computer implementation.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-12 are rejected under 35 U.S.C. 103 as being unpatentable over in view of O'Regan et al (US 20130063300 A1) hereafter O'Regan in view of Perkins et al (US 10665035 B1) hereafter Perkins in view of Cohen et al (US 20210343022 A1) hereafter Cohen
Regarding claim 1, O'Regan teaches a computer-implemented method of identifying and classifying infrastructure equipment assets along routes in a geographic region by analyzing a digital representation of the geographic region using artificial intelligence (AI) and computer image processing and analysis, the method comprising: receiving, by a virtual audit application stored in non-transitory memory of a computer system and executable by a processor of the computer system, an indication of a type of equipment to be audited and a geographic region (Para 0028, the location data for each of the respective tracking devices including a device identifier); determining, by the virtual audit application, based on a location map database, geographic coordinate information of a route in the geographic region (Para 0028, the method for tracking fluid spills when moving with a water flow includes the steps of obtaining location and movement data for each of a number of tracking devices from a positioning satellite data repository, the location data for each of the respective tracking devices including a device identifier, a geographic location and velocity of a tracking device, and a timestamp for each geographic location and velocity received); identifying, by the virtual audit application, from a geographic image database, a polyline representative of the route based on the geographic coordinate information (Para 0049, Typically, spatial data is stored in the form of points, polylines, polygons, vectors, imagery, or some other shape); traversing, by the virtual audit application, a plurality of points on the polyline (Para 0068, standard data animation controls in the map module 232 of the GIS module 113 allow a user to visualize the exact travel path of the fluid spill over a defined time period).
O'Regan does not appear to explicitly teach retrieving, by the virtual audit application, from the geographic image database comprising pre-existing street view images captured along roads, based on geographic coordinate information associated with the plurality of points, a plurality of images of street views, each associated with a respective one of the plurality of points; analyzing, by the virtual audit application, using one or more machine learning (ML) models, the plurality of images to identify and classify assets associated with the equipment, wherein the analyzing comprises adjusting views of the plurality of images for use by the one or more ML models; generating, by the virtual audit application, a report that includes information associated with the identified assets and references to images of the identified assets; and initiating, by the virtual audit application, based on the information in the report, an action associated with at least one of a record update, a record verification, or an infrastructure change recommendation.
In analogous art, Perkins teaches retrieving, by the virtual audit application, from the geographic image database comprising pre-existing street view images (Fig. 5 shows a street view image) captured along roads (Para 7, The structure may be a street scene), based on geographic coordinate information associated with the plurality of points, a plurality of images of street views, each associated with a respective one of the plurality of points (Para 17, The process includes acquiring a plurality of digital images, still frames and/or video images of the site, the structure, or both the site and the structure, with each of the digital images including one or more reference objects positioned on or about the site, the structure, or both); analyzing, by the virtual audit application the plurality of images to identify and classify assets associated with the equipment (Para 23, the process can electronically stitching the digital images together based on the identified reference objects and/or features in the second three-dimensional point cloud, electronically annotating the second three-dimensional point cloud with actual dimensions and other information specifically relating to the site and/or the structure)(“site and/or the structure” teaches “assets associated with the equipment”), generating, by the virtual audit application, a report that includes information associated with the identified assets and references to images of the identified assets (Para 28, pseudo-three-dimensional view of the structure which the user can move through using a computer or mobile device)(“view of the structure” teaches “a report”); and initiating, by the virtual audit application, based on the information in the report, an action associated with at least one of a record update, a record verification, or an infrastructure change recommendation (Para 29, it is possible to annotate the as-built survey model with real-world lengths, widths, heights, areas, volumes, etc., and overlay those dimensions on each of the digital images)(“annotate” is a species of “record update”). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan to include the teaching of Perkins. One of ordinary skill in the art would be motivated to implement this modification in order to provide manual record updates, as taught by Perkins (Para 29, points of interest on the structure are manually defined).
O'Regan in view of Perkins does not appear to explicitly teach analyzing, by the virtual audit application, using one or more machine learning (ML) models, wherein the adjusting comprises iteratively processing images using the one or more ML models and adjusting view parameters based on results of the ML processing, wherein the iterative processing includes at least: processing an image using a first ML model to identify a reference feature; adjusting at least one view parameter based on location of the identified reference feature to generate an adjusted image; and processing the adjusted image using a second ML model to identify and classify the assets.
In analogous art, Cohen teaches analyzing, by the virtual audit application, using one or more machine learning (ML) models, wherein the adjusting comprises iteratively processing images using the one or more ML models and adjusting view parameters based on results of the ML processing, wherein the iterative processing includes at least: processing an image using a first ML model to identify a reference feature (Para 0026, the object detection 126 may comprise an ROI associated with the object's representation in an image); adjusting at least one view parameter based on location of the identified reference feature to generate an adjusted image (Para 0029, determine a subset of the point clouds to associate with each object detection 126 generated by the first ML model 124, and/or translate coordinates of the subset from a sensor coordinate space to an image space and/or a modified image space where a “z-axis” of the image space extends through a center of an ROI of the object detection and the “x-” and “y-” axes); and processing the adjusted image using a second ML model to identify and classify the assets (Para 0060, classification is used as a discriminator, the point-object associated may be determined based on the point having a classification which corresponds with the image detection classification). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan in view of Perkins to include the teaching of Cohen. One of ordinary skill in the art would be motivated to implement this modification in order to detect object in an environment, as taught by Cohen (Para 0009, The techniques discussed herein relate to detecting an object in an environment and/or generating a three-dimensional region of interest (ROI) associated with such an object detection based on a plurality of sensor modalities).
Regarding claim 3, O'Regan in view of Perkins in view of Cohen teaches the method of claim 1, wherein the retrieving the plurality of images comprises: searching, by the virtual audit application, the geographic image database, for a most recent image in a proximity of an individual point of the plurality of points (O'Regan, Para 0123, The trajectory map may also display an aerial image of the fluid spill and locations of the tracking devices, where the tracking device locations are synchronized to the aerial image based on the timestamps of the locations).
Regarding claim 4, O'Regan in view of Perkins in view of Cohen teaches the method of claim 3, wherein the searching for the most recent image in the proximity of the individual point comprises: retrieving, by the virtual audit application, from the geographic image database, timestamp information of images associated with neighboring points of the individual point on the polyline; and comparing the timestamp information of the images associated with the neighboring points to select the most recent image in the proximity of the individual point (O'Regan, Para 0123, The trajectory map may also display an aerial image of the fluid spill and locations of the tracking devices, where the tracking device locations are synchronized to the aerial image based on the timestamps of the locations).
Regarding claim 5, O'Regan in view of Perkins in view of Cohen teaches the method of claim 4, wherein the neighboring points of the individual point are based on intersection points of a grid overlaid on the individual point (O'Regan, Para 0049, For example, geographic coordinates and associated metadata for points of interest may be stored in a point map layer).
Regarding claim 6, O'Regan in view of Perkins in view of Cohen teaches the method of claim 1, wherein: the adjusting the views of the plurality of images comprises: adjusting at least one of a camera bearing or a field of view (FOV) of an individual image of the plurality of images to generate a first adjusted image, and the analyzing the plurality of images further comprises: processing the first adjusted image using a first ML model of the ML models to identify at least one of a pole or one or more assets of the assets associated with the equipment in the first adjusted image (Perkins, Para 9, The process can also include the steps of electronically selecting common feature points in two or more of the digital images, calculating camera positions, orientations, and distortions, and then generating a first three-dimensional point cloud of the site and/or the structure in the digital images). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan to include the teaching of Perkins. One of ordinary skill in the art would be motivated to implement this modification in order to provide manual record updates, as taught by Perkins (Para 29, points of interest on the structure are manually defined).
Regarding claim 7, O'Regan in view of Perkins in view of Cohen teaches the method of claim 6, wherein: the adjusting the views of the plurality of images further comprises: adjusting at least one of a camera bearing or an FOV of the first adjusted image based on the at least one of the pole or the one or more assets identified by the first ML model to generate a second adjusted image, and the analyzing the plurality of images comprises: processing the second adjusted image using a second ML model of the ML models to identify the one or more assets in the second adjusted image (Para 9, Additionally, the process can include densifying, leveling and orienting the first three-dimensional point cloud based on the identified reference objects, on the identified features, on one or more identified tie points, or a combination thereof to generate a second three-dimensional point cloud). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan to include the teaching of Perkins. One of ordinary skill in the art would be motivated to implement this modification in order to provide manual record updates, as taught by Perkins (Para 29, points of interest on the structure are manually defined).
Regarding claim 8, O'Regan in view of Perkins in view of Cohen teaches the method of claim 6, wherein: the adjusting the views of the plurality of images further comprises: adjusting the camera bearing of the individual image to generate the first adjusted image to provide a left-side (LS) view or a right-side (RS) view with respect to a respective one of the plurality of points along the polyline; and adjusting at least one of a camera bearing or an FOV of the first adjusted image with respect to the identified pole to generate a second adjusted image (Para 26, The right angle of the structure and/or the reference object 400 can be located (step 900) and the north facing pole can be located to orient the model); and adjusting at least one of a camera bearing, a pitch, or an FOV of the second adjusted image with respect to the one or more identified assets to generate a third adjusted image, and the analyzing the plurality of images further comprises: processing the third adjusted image, using a second ML model of the one or more ML models, to output an indication of the one or more assets in the third adjusted image and a classification of each of the one or more assets (Para 26, Then similar to FIG. 8, tie points can be added (step 904) to denote the Z orientation (step 906) and the X orientation (step 908) of the reference object 400. Then the inventive process and system 100 re-optimizes (step 910) and levels the point cloud based on the reference objects). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan to include the teaching of Perkins. One of ordinary skill in the art would be motivated to implement this modification in order to provide manual record updates, as taught by Perkins (Para 29, points of interest on the structure are manually defined).
Regarding claim 9, O'Regan in view of Perkins in view of Cohen teaches the method of claim 8, wherein: the analyzing the plurality of images further comprises: computing, based on the second adjusted image, an asset enclosing bounding box to enclose the one or more assets in the second adjusted image, and the adjusting the at least one of the camera bearing, the pitch, or the FOV of the second adjusted image is further with respect to the computed asset enclosing bounding box (Para 18, one or more two-dimensional scaling reference objects 400 and/or one or more three-dimensional scaling reference objects 300 may be placed within the field of view of the camera). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan to include the teaching of Perkins. One of ordinary skill in the art would be motivated to implement this modification in order to provide manual record updates, as taught by Perkins (Para 29, points of interest on the structure are manually defined).
Regarding claim 10, O'Regan in view of Perkins in view of Cohen teaches the method of claim 1, wherein the information associated with identified assets in the report comprises, for each of the identified assets, at least one of: a classification and a corresponding confidence score for the respective identified asset, geographic coordinate information associated with the respective identified asset, a camera bearing associated with the respective identified asset, an FOV associated with the respective identified asset, or a pitch associated with the respective identified asset (Perkins, Para 9, The process can also include the steps of electronically selecting common feature points in two or more of the digital images, calculating camera positions, orientations, and distortions, and then generating a first three-dimensional point cloud of the site and/or the structure in the digital images). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan to include the teaching of Perkins. One of ordinary skill in the art would be motivated to implement this modification in order to provide manual record updates, as taught by Perkins (Para 29, points of interest on the structure are manually defined).
Regarding claim 11, O'Regan in view of Perkins in view of Cohen teaches the method of claim 1, wherein the analyzing the plurality of images further comprises: assessing, by the virtual audit application, a condition of at least one of the identified assets (Perkins, Para 9, The process can also include the steps of electronically selecting common feature points in two or more of the digital images, calculating camera positions, orientations, and distortions, and then generating a first three-dimensional point cloud of the site and/or the structure in the digital images). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan to include the teaching of Perkins. One of ordinary skill in the art would be motivated to implement this modification in order to provide manual record updates, as taught by Perkins (Para 29, points of interest on the structure are manually defined).
Regarding claim 12, O'Regan in view of Perkins in view of Cohen the method of claim 1, wherein the identified assets comprise at least one of a power supply, a splice enclosure, telephony equipment, a tap, an amplifier, or a transformer (O'Regan, Para 0083, if a low battery notification has been received from a tracking device, the GIS map display discussed below may depict a low battery notification adjacent to a representation of the tracking device).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over in view of O'Regan in view of Perkins in view of Cohen in view of Hiramoto et al (US 20190087993 A1) hereafter Hiramoto
Regarding claim 2, O'Regan in view of Perkins in view of Cohen teaches the method of claim 1, as shown above. O'Regan in view of Perkins in view of Cohen does not appear to explicitly teach further comprising: retrieving, by the virtual audit application, from the geographic image database, second geographic coordinate information of a plurality of second points on the polyline; and computing, by the virtual audit application, the geographic coordinate information of at least some of the plurality of points on the polyline based on an interpolation of the second geographic coordinate information of the plurality of second points.
In analogous art, Hiramoto retrieving, by the virtual audit application, from the geographic image database, second geographic coordinate information of a plurality of second points on the polyline; and computing, by the virtual audit application, the geographic coordinate information of at least some of the plurality of points on the polyline based on an interpolation of the second geographic coordinate information of the plurality of second points (Para 0024, The location and shape of the road are the location and shape of a polyline that connects nodes and shape interpolation points which are indicated by the map information). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify O'Regan in view of Perkins in view of Cohen to include the teaching Hiramoto. One of ordinary skill in the art would be motivated to implement this modification in order to provide a map display system, as taught by Hiramoto (Abs, There are provided a map display system and a map display program that can arrange characters along a road so as to improve appearance and readability).
Claims 13-15 are rejected under 35 U.S.C. 103 as being unpatentable over in view of Perkins in view of Jayawickrema et al (US 20200298498 A1) hereafter Jayawickrema
Regarding claim 13, Perkins teaches a computer-implemented method of evaluating and updating a machine learning (ML) model for virtual auditing of infrastructure equipment along routes based on rules associated with physical installation characteristics of the infrastructure equipment, the method comprising: receiving, by a virtual audit application stored in non-transitory memory of a computer system and executable by a processor of the computer system, a type of equipment to be audited and a geographic region (Para 17, The process includes acquiring a plurality of digital images, still frames and/or video images of the site, the structure, or both the site and the structure, with each of the digital images including one or more reference objects positioned on or about the site, the structure, or both); retrieving, by the virtual audit application, a plurality of images of the geographic region from a geographic image database comprising pre-existing street view images captured along roads(Para 7, The structure may be a street scene); analyzing, by the virtual audit application, using one or more ML models asset identification, the plurality of images to identify and classify assets associated with the equipment(Para 23, the process can electronically stitching the digital images together based on the identified reference objects and/or features in the second three-dimensional point cloud, electronically annotating the second three-dimensional point cloud with actual dimensions and other information specifically relating to the site and/or the structure)(“site and/or the structure” teaches “assets associated with the equipment”).
Perkins does not appear to explicitly teach using one or more ML models trained for autonomous infrastructure; wherein the analyzing comprises: processing a first image of the plurality of images using a first ML model of the one or more ML models to identify a first asset of the assets; determining, by an ML model training application stored in the non-transitory memory of the computer system and executable by the processor of the computer system, that the first ML model fails to identify one or more other assets associated with the equipment based on rules encoding physical coexistence relationships between asset types, wherein the rules specify that detection of a first asset type requires presence of a second asset type associated with characteristics of the equipment; and updating, by the ML model training application, based on the determining, one or more parameters of the first ML model.
In analogous art, Jayawickrema teaches using one or more ML models trained for autonomous infrastructure(Para 0015, ADR 110 can create predictive models); processing a first image of the plurality of images using a first ML model of the one or more ML models to identify a first asset of the assets (Para 0015, ADR 110 can create predictive models that, when compared to build images, can recognize recoater defects without need for labeled training sets); determining, by an ML model training application stored in the non-transitory memory of the computer system and executable by the processor of the computer system, that the first ML model fails to identify one or more other assets associated with the equipment based on rules encoding physical coexistence relationships between asset types (Para 0016, users determine whether flagged indications are false, the models can be updated to incorporate the user determinations), wherein the rules specify that detection of a first asset type requires presence of a second asset type associated with characteristics of the equipment (Para 0036, Action is taken based on a catalog of predefined rules that are applied to the virtual depiction. The rules can specify the quantity, size and penetration of indications dependent on the part being built); and updating, by the ML model training application, based on the determining, one or more parameters of the first ML model (Para 0016, users determine whether flagged indications are false, the models can be updated to incorporate the user determinations). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Perkins to include the teaching of Jayawickrema. One of ordinary skill in the art would be motivated to implement this modification in order to increase monitoring performance, as taught by Jayawickrema (Para 0016, In accordance with embodiments, recoater automated monitoring system 100 performance can be improved with feedback)
Regarding claim 14, Perkins in view of Jayawickrema teaches the method of claim 13, wherein the updating the one or more parameters of the first ML model comprises: training, by the ML model training application, the first ML model to further identify the one or more other assets determined based on the rules associated with the characteristics of the equipment (Jayawickrema, Para 0015, ADR 110 can create predictive models that, when compared to build images, can recognize recoater defects without need for labeled training sets). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Perkins to include the teaching of Jayawickrema. One of ordinary skill in the art would be motivated to implement this modification in order to increase monitoring performance, as taught by Jayawickrema (Para 0016, In accordance with embodiments, recoater automated monitoring system 100 performance can be improved with feedback).
Regarding claim 15, Perkins in view of Jayawickrema teaches the method of claim 13, wherein the rules associated with the characteristics of the equipment comprises at least one of: an indication of a coexistence between a first type of assets and a second type of assets for the equipment, or a comparison against an external data source having records of assets associated with the equipment (Jayawickrema, Para 0015, ADR 110 can create predictive models that, when compared to build images, can recognize recoater defects without need for labeled training sets). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Perkins to include the teaching of Jayawickrema. One of ordinary skill in the art would be motivated to implement this modification in order to increase monitoring performance, as taught by Jayawickrema (Para 0016, In accordance with embodiments, recoater automated monitoring system 100 performance can be improved with feedback).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over in view of Perkins in view of Jayawickrema further in view of Asmari et al (US 20210020073 A1) hereafter Asmari
Regarding claim 16, Perkins in view of Jayawickrema teaches the method of claim 13, as taught above. Perkins in view of Jayawickrema does not appear to explicitly teach wherein the one or more other assets determined based on the rules comprise at least one of a power supply, a splice enclosure, telephony equipment, a tap, an amplifier, or a transformer.
In analogous art, Asmari teaches wherein the one or more other assets determined based on the rules comprise at least one of a power supply, a splice enclosure, telephony equipment, a tap, an amplifier, or a transformer (Para 0009, A machine-learning algorithm detects target assets—e.g., utility poles, transformers, etc.—in the video feed collected by the vehicle-mounted cameras and other sensors). It would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify Perkins in view of Jayawickrema to include the teaching of Asmari. One of ordinary skill in the art would be motivated to implement this modification in order to perform asset identification, as taught by Asmari (A system for asset identification and mapping is configured to information related to a plurality of objects with at least one sensor).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Brooks Hale whose telephone number is 571-272-0160. The examiner can normally be reached 9am to 5pm est.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sanjiv Shah can be reached on (571) 272-4098. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/B.T.H./Examiner, Art Unit 2166
/SANJIV SHAH/Supervisory Patent Examiner, Art Unit 2166
Prosecution Insights