Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to amendment filed 03/31/2026 in which claims 1-21 are pending.
Response to Arguments
Applicant’s arguments, see pages 8-25, filed 03/31/2026, with respect to the rejections of claims have been fully considered and amended claims are moot in view of new grounds of rejection made in view of Choi et al.(WO 2023182855 A1 ).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 21 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 21 recites the limitation "the optical timing device". There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 3, 5-7, 10, 12 , 14-16, 19 are rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached).
Regarding claim 1, Currin discloses a method for automatically detecting at least one anomaly inside of a conduit in real-time computing (Para [0074] teaches sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main. This approach is effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults. Para[0109] teaches during or after the loading of an inspection video (or other form of digital inspection data), if highly accurate geospatial locations have been collected for the manholes at the ends of the inspected sewer segment, the application may automatically derive and assign highly accurate geospatial coordinates to one or more or all of the points of the length of the pipe segment between the two manholes, as well as assigning corresponding geospatial locations to one or more or all of the frames of the inspection video. This enables faults, lateral connections, priority repairs, etc. to all be shown as symbols overlaid on a base mapping layer), comprising steps of: moving an optical imaging device of a system inside of the conduit (para[0015] leading method of sewer pipe inspection today is to insert a video camera connected to a closed-circuit television (CCTV) system down a sewer manhole, while inspection personnel narrate the conditions in real time as they see them, noting cracks, leaks, roots, FOG, and other blockages. Run his own camera down the line, and try to find the fault noted on the plat map. Para[0074] teaches of sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main. This approach is effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults. The video camera can also be run off of a sewer main to capture the inside of a lateral that connects the main to a residence or business) viewing at least one anomaly inside of the conduit with the optical imaging device in real-time (para[0083] teaches a user may also be able to view images or video that a contractor is capturing during an inspection, in real time while it is being captured, para[0122] teaches worker generally views the data in real time and records observations. Para[0127] teaches the inspection data may be uploaded in real time, such that inspection data (e.g., inspection video data) is available through server platform 110 in real time as the inspection is taking place. ); outputting a live video stream by the optical imaging device with the at least one anomaly to a control interface of the system (para[0077] teaches an original video 620 may be received by importer module 610. Importer module 610 processes video 620 to prepare it for storage and access); executing an anomaly detection program, by a controller of the system, from a non- transitory computer readable medium (Para[0173] teaches the removable storage medium 580 is a non-transitory computer-readable medium having stored thereon computer executable code (i.e., software) and/or data. The computer software or data stored on the removable storage medium 580 is read into the system 550 for execution by the processor 560) in response to the at least one anomaly being viewed by the optical imaging device from the live video stream, wherein the controller is caused to: and automatically indicate the at least one detected anomaly on the live video stream (Para[0015] teaches inspection personnel narrate the conditions in real time as they see them, noting cracks, leaks, roots, FOG, and other blockages. Para[0127] teaches the inspection data may be uploaded in real time, such that inspection data (e.g., inspection video data) is available through server platform 110 in real time as the inspection is taking place. This provides the ability for real-time collaboration during an inspection, for example, between the workers in the field and consulting engineers in their home or office).
Currin does not explicitly disclose viewing at least one anomaly inside of the conduit with the optical imaging device in real-time at a single viewpoint; automatically detect the at least one anomaly with a machine learning protocol of the anomaly detection program; output the at least one detected anomaly to the control interface, wherein the machine learning protocol is configured to process the live video stream in real-time as the optical imaging device moves inside of the conduit. However Michaud discloses automatically detect the at least one anomaly with a machine learning protocol of the anomaly detection program (Para[0040], Para[0041] –[0043] teaches machine learning (e.g., machine learning technology, such as convolutional neural networks models (CNNs), deep belief networks models (DBNs), etc.) to identify the sewer specific characteristics . The CNNs can include a plurality of hyperparameters that can each be tuned to obtain the model having the best performance in identifying and categorizing the sewer specific characteristics of the sewer line (or the absence thereof) from the associated images and sewer inspection metadata. Para[0049] & Fig. 2 teaches the sewer specific characteristics of the sewer line could also include structural defects, such as, for example, cracks in the sewer line, fractures in the sewer line, holes in the sewer line, deformations in the sewer line, collapses of sections of the sewer line, etc. para[0045] teaches characteristic identification and categorization system can use a single general CNN with finely tuned hyperparameters to obtain the model performing the image analysis and predict (i.e. identify and categorize) the different sewer specific characteristics depicted in the images. Para[0062] identification module 44 is a characteristic CNN module using one or more CNNs 42 that are associated with the sewer specific characteristics to be inspected in the associated sewer line and use the models obtained from the selected CNNs 42 to identify and categorize one or more characteristics of the sewer line depicted in the images 27 (or predict the absence thereof)); output the at least one detected anomaly to the control interface (Para[0073]& Fig. 2 teaches once the identification module 44 has made the prediction relative to the identification/categorization of the characteristics of the sewer line, the identification data 46 generated by the identification module 44 can be transmitted to the report production module 32. In the embodiment shown, the report production module 32 is configured to transmit the identification data 46 to an analyst computing device 30, for example over a communication network 14, and is configured to provide a user interface which can be displayed on the analyst computing device 30 to display at least a portion of the identification data 46 to the analyst, including images 27 from the sewer inspection data 25 and the identified/categorized characteristics (or the absence thereof), for validation and/or for selection of the most likely identified/categorized characteristics by the analyst, during the analysis of the images 27 from the sewer inspection data 25). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin with the method for automated inspection of sewer line using machine learning of Michaud in order to provide a system for characteristic identification and categorization system process the sewer inspection data and proceed with the prediction of the presence of the sewer specific characteristics and categorization of the characteristics.
Currin in view of Michaud does not explicitly disclose viewing at least one anomaly inside of the conduit with the optical imaging device in real-time at a single viewpoint; wherein the machine learning protocol is configured to process the live video stream in real-time as the optical imaging device moves inside of the conduit. However Choi disclose viewing at least one anomaly inside of the conduit with the optical imaging device in real-time at a single viewpoint (para[0008] teaches , the image converter may generate the at least one predicted image data to include at least one region of interest (ROI).para[0038] teaches the image conversion unit 120 May include at least one region of interest (ROI) in the prediction image data. The at least one region of interest may include, for example, a specific region assuming that a crack has occurred in the pipe of the water and sewage facility); wherein the machine learning protocol is configured to process the live video stream in real-time as the optical imaging device moves inside of the conduit (para[0018] teaches the method for detecting defects in water supply and sewage facilities according to various embodiments disclosed in this document and the defect detection system supporting the method detect structural defects inside water supply and sewerage facilities in real time from images taken of the inside of the water supply and sewage facilities, thereby reducing user dependence due to post-defect detection can be reduced. Para[0041] teaches the defect detection module 140 May determine whether a defect is included in the real-time image data obtained from the water and sewage facility. For example, when the defect detection module 140 receives real-time image data using an exploration device (e.g. an intelligent pig)Para[0048] teaches to operation 240, the defect detection system 100 May determine whether a defect is included in the real-time image data obtained from the water and sewage facility. Para[0062]-[0065] teaches to step S320, the water and sewage facility 101 May acquire real-time image data inside the facility. The real-time image data may be, for example, an image captured using an exploration device (e.g. an intelligent pig).Referring to step S321, the water and sewage facility 101 May transmit the real-time image data obtained in step S321 to the defect detection system 100.Referring to step S322, the defect detection system 100 May receive real-time image data from the water and sewage facility 101.Referring to operation S323, the defect detection system 100 May determine whether a defect is included in the real-time image data received in operation S322). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin in view of Michaud with the method possible to provide a water and sewage facility defect detection of Choi in order to provide a system capable of detecting structural defects inside the water and sewage facility in real time from an image of the inside of the water and sewage facility, and a defect detection supporting the same.
Regarding claim 3, Currin in view of Michaud and Choi discloses the method of claim 1, Currin further discloses wherein the conduit assessor is loaded with pipe, lateral, and manhole assessment coding guidelines (Para[0129] teaches the server application may automatically process the inspection data to prepare a NASSCO report, Para[0147] teaches layers may comprise a layer for mainline pipe rating, a layer for pipe inspections, a layer for manholes, a layer for lateral pipes, a layer for mainline pipes, and/or a layer for contracts. Para[0163] teaches server platform 110 may be compliant, not only with PACP, but also with the metadata requirements for inspections under the Lateral Assessment and Certification Program (LACP)and Manhole Assessment and Certification Program (MACP)).
Michaud further discloses wherein the step of automatically detecting the at least one anomaly from the machine learning protocol further comprises: determining a type of anomaly of the at least one anomaly by a conduit assessor of the machine learning protocol (Para[0040] teaches sewer specific characteristics of the sewer line can include attributes (or observation types) which are relevant to the description, condition and/or performance of the sewer line. For example and without being limitative, in an embodiment, the sewer specific characteristics of the sewer line can include attributes described in the Pipeline Assessment Certification Program® (PACP®) from NASSCO. The sewer specific characteristics of the sewer line can relate to different types of elements requiring operation and/or maintenance such as deposits in the sewer line, roots in the sewer line, infiltration in the sewer line, obstacles/obstructions in the sewer line, vermin in the sewer line, etc. The sewer specific characteristics of the sewer line could also include structural defects, such as, for example, cracks in the sewer line, fractures in the sewer line, holes in the sewer line, deformations in the sewer line, collapses of sections of the sewer line, etc. The sewer specific characteristics of the sewer line could further include construction features such as the type and condition of the taps of the sewer line, the type of access points of the sewer line, etc.). Motivation to combine as indicted in claim 1.
Regarding claim 5, Michaud further discloses the method of claim 1, further comprising: intercepting the video stream, by a video interceptor, between the optical imaging device and the control interface (para[0060] & Fig. 1 teaches the sewer inspection data 25 can be uploaded to a user computing device 36 in data communication with the inspection data database 24, following an inspection of a sewer line, in anticipation of the preparation of an inspection report. For example and without being limitative, the sewer inspection data 25 can be uploaded to the user computing device 36 from an inspection device (38) used to perform the inspection and used in the field to acquire the sewer inspection data 25, such as, for example and without being limitative, an inspection robot, or the like.). Motivation to combine as indicated in claim 1
Regarding claim 6, Michaud further discloses the method of claim 5, wherein the step of intercepting the video stream further comprises: outputting the video stream, by the video interceptor, to the controller; outputting the video stream, by the controller, to a dedicated monitor separate from the control interface; and indicating the at least one detected anomaly on the dedicated monitor separate from the control interface (Para[0070]-[0071] teaches the sewer inspection data 25 can be uploaded from an inspection device 38 to the user computing device 36 which further operates to upload the sewer inspection data 25 to the characteristic identification and categorization system 20. The user interface can be displayed on a display screen of the user computing device 36, to allow a user to upload the sewer inspection data 25 from the inspection device 38 to the user computing device 36 and subsequently upload the sewer inspection data 25 to the characteristic identification and categorization system 20. In an embodiment, the user computing device 36 can be used to edit or complete the sewer inspection data 25, before performing the upload to the characteristic identification and categorization system 20). Motivation to combine as indicated in claim 1.
Regarding claim 7, Michaud further discloses the method of claim 1, further comprising: updating the machine learning protocol of the anomaly detection program by a cloud-based repository (Para[0067] Once the models obtained from the CNNs 42 yielding the most accurate inspection results have been determined in the training environment, the identification module 44 including the models obtained from the CNNs 42 yielding the most accurate inspection results (i.e. the models obtained from the CNNs having the best CNN type and hyperparameter configuration) is imported onto the production servers 62 of a production environment. For example and without being limitative, in an embodiment, the production servers 62 can be commercial servers of a cloud computing service ). Motivation to combine as indicated in claim 1.
Regarding claim 10, Currin discloses a system for automatically detecting at least one anomaly inside of a conduit in real-time computing (Para[0109] teaches during or after the loading of an inspection video (or other form of digital inspection data), if highly accurate geospatial locations have been collected for the manholes at the ends of the inspected sewer segment, the application may automatically derive and assign highly accurate geospatial coordinates to one or more or all of the points of the length of the pipe segment between the two manholes, as well as assigning corresponding geospatial locations to one or more or all of the frames of the inspection video. This enables faults, lateral connections, priority repairs, etc. to all be shown as symbols overlaid on a base mapping layer), comprising: an optical imaging device outputting a live video stream (para [0074] teaches sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main. This approach is effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults); a controller operatively in communication with the optical imaging device (para[0083] teaches a user may also be able to view images or video that a contractor is capturing during an inspection, in real time while it is being captured, para[0122] teaches worker generally views the data in real time and records observations. Para[0127] teaches the inspection data may be uploaded in real time, such that inspection data (e.g., inspection video data) is available through server platform 110 in real time as the inspection is taking place; a control interface operatively in communication with the optical imaging device and the controller (para[0077] teaches an original video 620 may be received by importer module 610. Importer module 610 processes video 620 to prepare it for storage and access); the controller is instructed to automatically detect the at least one anomaly inside of the conduit in response to the optical imaging device when viewing the at least one anomaly inside of the conduit from the live video stream (Para[0015] teaches inspection personnel narrate the conditions in real time as they see them, noting cracks, leaks, roots, FOG, and other blockages. Para[0127] teaches the inspection data may be uploaded in real time, such that inspection data (e.g., inspection video data) is available through server platform 110 in real time as the inspection is taking place. This provides the ability for real-time collaboration during an inspection, for example, between the workers in the field and consulting engineers in their home or office).
Currin does not explicitly disclose an optical imaging device outputting a live video stream at a single viewpoint; an anomaly detection program having a machine learning protocol and is stored on a non-transitory computer readable medium that is executable by the controller; wherein when the controller executes the machine learning protocol of the anomaly detection program, the controller is instructed to automatically detect the at least one anomaly inside of the conduit in real time in response to the optical imaging device viewing the at least one anomaly inside of the conduit from the live video stream.. However Michaud discloses an anomaly detection program having a machine learning protocol and is stored on a computer readable medium that is executable by the controller; wherein when the controller executes the machine learning protocol of the anomaly detection program Para[0040], Para[0041] –[0043] teaches machine learning (e.g., machine learning technology, such as convolutional neural networks models (CNNs), deep belief networks models (DBNs), etc.) to identify the sewer specific characteristics . The CNNs can include a plurality of hyperparameters that can each be tuned to obtain the model having the best performance in identifying and categorizing the sewer specific characteristics of the sewer line (or the absence thereof) from the associated images and sewer inspection metadata. Para[0049] & Fig. 2 teaches the sewer specific characteristics of the sewer line could also include structural defects, such as, for example, cracks in the sewer line, fractures in the sewer line, holes in the sewer line, deformations in the sewer line, collapses of sections of the sewer line, etc. para[0045] teaches characteristic identification and categorization system can use a single general CNN with finely tuned hyperparameters to obtain the model performing the image analysis and predict (i.e. identify and categorize) the different sewer specific characteristics depicted in the images. Para[0062] identification module 44 is a characteristic CNN module using one or more CNNs 42 that are associated with the sewer specific characteristics to be inspected in the associated sewer line and use the models obtained from the selected CNNs 42 to identify and categorize one or more characteristics of the sewer line depicted in the images 27 (or predict the absence thereof). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin with the method for automated inspection of sewer line using machine learning of Michaud in order to provide a system for characteristic identification and categorization system process the sewer inspection data and proceed with the prediction of the presence of the sewer specific characteristics and categorization of the characteristics.
Currin in view of Michaud does not explicitly disclose an optical imaging device outputting a live video stream at a single viewpoint, the controller is instructed to automatically detect the at least one anomaly inside of the conduit in real time in response to the optical imaging device viewing the at least one anomaly inside of the conduit from the live video stream. However Choi disclose an optical imaging device outputting a live video stream at a single viewpoint (para[0008] teaches , the image converter may generate the at least one predicted image data to include at least one region of interest (ROI).para[0038] teaches the image conversion unit 120 May include at least one region of interest (ROI) in the prediction image data. The at least one region of interest may include, for example, a specific region assuming that a crack has occurred in the pipe of the water and sewage facility); the controller is instructed to automatically detect the at least one anomaly inside of the conduit in real time in response to the optical imaging device viewing the at least one anomaly inside of the conduit from the live video stream (para[0018] teaches the method for detecting defects in water supply and sewage facilities according to various embodiments disclosed in this document and the defect detection system supporting the method detect structural defects inside water supply and sewerage facilities in real time from images taken of the inside of the water supply and sewage facilities, thereby reducing user dependence due to post-defect detection can be reduced. Para[0041] teaches the defect detection module 140 May determine whether a defect is included in the real-time image data obtained from the water and sewage facility. For example, when the defect detection module 140 receives real-time image data using an exploration device (e.g. an intelligent pig)Para[0048] teaches to operation 240, the defect detection system 100 May determine whether a defect is included in the real-time image data obtained from the water and sewage facility. Para[0062]-[0065] teaches to step S320, the water and sewage facility 101 May acquire real-time image data inside the facility. The real-time image data may be, for example, an image captured using an exploration device (e.g. an intelligent pig).Referring to step S321, the water and sewage facility 101 May transmit the real-time image data obtained in step S321 to the defect detection system 100.Referring to step S322, the defect detection system 100 May receive real-time image data from the water and sewage facility 101.Referring to operation S323, the defect detection system 100 May determine whether a defect is included in the real-time image data received in operation S322).It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin in view of Michaud with the method possible to provide a water and sewage facility defect detection of Choi in order to provide a system capable of detecting structural defects inside the water and sewage facility in real time from an image of the inside of the water and sewage facility, and a defect detection supporting the same.
Regarding claim 12, Currin further discloses the system of claim 10, wherein the machine learning protocol further comprises: a conduit assessor operatively in communication with a video transcoding architecture of the anomaly detection program (Para[0059] & Fig. 4 teaches shape file loader module 410 determines, to the extent possible, the coordinate or projection system associated with the received shape file(s). Shape file loader module 410 then recasts the geometry of the shape file dataset 420 of the shape file(s) into WGS 84 format, resulting in one or more shape file objects 430. Shape file objects 430 comprise infrastructure data, including infrastructure asset locations in WGS 84 format) and configured with pipe, lateral, and manhole assessment coding guidelines (Para[0129] teaches the server application may automatically process the inspection data to prepare a NASSCO report, Para[0147] teaches layers may comprise a layer for mainline pipe rating, a layer for pipe inspections, a layer for manholes, a layer for lateral pipes, a layer for mainline pipes, and/or a layer for contracts. Para[0163] teaches server platform 110 may be compliant, not only with PACP, but also with the metadata requirements for inspections under the Lateral Assessment and Certification Program (LACP)and Manhole Assessment and Certification Program (MACP)).
Regarding claim 14, Michaud discloses the method of claim 10, further comprising: a video interceptor operatively in communication with the optical imaging device and the control interface and configured to output the live video stream to the controller (Para[0060] & Fig. 1 teaches the sewer inspection data 25 can be uploaded to a user computing device 36 in data communication with the inspection data database 24, following an inspection of a sewer line, in anticipation of the preparation of an inspection report. For example and without being limitative, the sewer inspection data 25 can be uploaded to the user computing device 36 from an inspection device (38) used to perform the inspection and used in the field to acquire the sewer inspection data 25, such as, for example and without being limitative, an inspection robot, or the like.). Motivation to combine as indicated in claim 10.
Regarding claim 15, Michaud discloses the method of claim 14, further comprising: a dedicated monitor operatively in communication with the controller and configured to indicate the at least one detected anomaly (para[0073] teaches the report production module 32 is configured to transmit the identification data 46 to an analyst computing device 30, for example over a communication network 14, and is configured to provide a user interface which can be displayed on the analyst computing device 30 to display at least a portion of the identification data 46 to the analyst, including images 27 from the sewer inspection data 25 and the identified/categorized characteristics (or the absence thereof), for validation and/or for selection of the most likely identified/categorized characteristics by the analyst, during the analysis of the images 27 from the sewer inspection data 25). Motivation to combine as indicated in claim 10.
Regarding claim 16, Michaud further discloses the method of claim 10, further comprising: a cloud-based repository component operatively in communication with the controller and configured to provide at least one update for the machine learning protocol (Para[0067] Once the models obtained from the CNNs 42 yielding the most accurate inspection results have been determined in the training environment, the identification module 44 including the models obtained from the CNNs 42 yielding the most accurate inspection results (i.e. the models obtained from the CNNs having the best CNN type and hyperparameter configuration) is imported onto the production servers 62 of a production environment. For example and without being limitative, in an embodiment, the production servers 62 can be commercial servers of a cloud computing service).
Regarding claim 19, Currin discloses a computer program product stored on a non-transitory computer readable media and executable by a controller of a system (Para[0173] teaches the removable storage medium 580 is a non-transitory computer-readable medium having stored thereon computer executable code (i.e., software) and/or data. The computer software or data stored on the removable storage medium 580 is read into the system 550 for execution by the processor 560, para[0176] teaches computer software or executable code may be transferred to system 550, para[0179]-[0180], [0187] ) for automatically detecting at least one anomaly inside of a conduit in real-time computing (Para [0074] teaches sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main. This approach is effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults, Para[0109] teaches during or after the loading of an inspection video (or other form of digital inspection data), if highly accurate geospatial locations have been collected for the manholes at the ends of the inspected sewer segment, the application may automatically derive and assign highly accurate geospatial coordinates to one or more or all of the points of the length of the pipe segment between the two manholes, as well as assigning corresponding geospatial locations to one or more or all of the frames of the inspection video. This enables faults, lateral connections, priority repairs, etc. to all be shown as symbols overlaid on a base mapping layer), the computer program product comprising instructions for: executing, by the controller, at least one anomaly being viewed by on a live video stream recorded by an optical imaging device of the system (para[0083] teaches a user may also be able to view images or video that a contractor is capturing during an inspection, in real time while it is being captured, para[0122] teaches worker generally views the data in real time and records observations. Para[0127] teaches the inspection data may be uploaded in real time, such that inspection data (e.g., inspection video data) is available through server platform 110 in real time as the inspection is taking place.); executing, by the controller, a second step to output the at least one detected anomaly to the control interface (Para[0080] teaches one or more modules of server platform 110 automatically prepare the uploaded video for hosting in the cloud, adding speed control, security control enhancements, and/or other processing to generate hosted, high-availability videos. The videos are automatically linked to the relevant, corresponding pipe segments and the originating manholes in the corresponding shape files that have been previously loaded and geospatially aligned by server platform 110. Clicking on a presented video icon, which may be overlaid on a virtual map at the location corresponding to the location of the asset that is the subject of the video represented by the icon, allows the city engineer to review that video (e.g., foot by foot). The video may be presented along with any observations made concerning the video. Additional attribute details can be added to any existing note about a fault or an observation. Geospatial links can be created for any note that links to the exact spot in the video footage and/or on the virtual map. Such links may then show a coded fault as a searchable or clickable symbol on the virtual map. New faults and observations can be noted and automatically linked to geospatial coordinates. Extended faults, such as root growth over a twenty foot section of pipe, can be geospatially linked to the virtual map in their full extent.); and executing, by the controller, a third step to automatically indicate the at least one detected anomaly on the live video stream (para[0070] teaches all faults identified within any inspection video or other inspection data stream can have geospatial coordinates associated with them. Using these highly accurate geospatial coordinates, faults and other items of interest can be provided in a mapping layer of a server application, which cannot be done at all when the measurements are simply in linear feet from a manhole. Nor can they be accurately done if the geospatial coordinates of the manholes have not first been established with high accuracy. For example, a pipe segment can be shown on a virtual map to have five critical faults within a short section of right-of-way that also suffers from some form of environmental contamination. Such knowledge may instantly inform the best option for needed repair , Para[0109] teaches (6) During or after the loading of an inspection video (or other form of digital inspection data), if highly accurate geospatial locations have been collected for the manholes at the ends of the inspected sewer segment, the application may automatically derive and assign highly accurate geospatial coordinates to one or more or all of the points of the length of the pipe segment between the two manholes, as well as assigning corresponding geospatial locations to one or more or all of the frames of the inspection video. This enables faults, lateral connections, priority repairs, etc. to all be shown as symbols overlaid on a base mapping layer. Geospatial locations can also be automatically assigned to other forms of sewer inspection, such as electromagnetic or any other kind of electrical or ultrasonic form of continuous pipeline inspection data. Para[0110] (7) The application may leverage geospatial fault locations and the time dimension against other attribute data in order to support more effective and more efficient sewer pipe management. For example, the ability of the application to automatically assign geospatial extents to CCTV-observed root growth or FOG areas of a sewer pipe allows an automatic virtual overlay of multiple years of observed problems to visually represent those stretches of sewer pipe that consistently need extra attention and cleaning. Para[0115] teaches the locations of all assets and/or faults in the query results window may automatically be shown on the application-provided virtual map).
Currin does not explicitly disclose a first step to automatically detect the at least one anomaly with a machine learning protocol of an anomaly detection program in real-time in response to the at least one anomaly being viewed by on a live video stream recorded by an optical imaging device of the system at a single viewpoint. However Michaud discloses a first step to automatically detect the at least one anomaly with a machine learning protocol of an anomaly detection program in response to the at least one anomaly Para[0040], Para[0041] –[0043] teaches machine learning (e.g., machine learning technology, such as convolutional neural networks models (CNNs), deep belief networks models (DBNs), etc.) to identify the sewer specific characteristics . The CNNs can include a plurality of hyperparameters that can each be tuned to obtain the model having the best performance in identifying and categorizing the sewer specific characteristics of the sewer line (or the absence thereof) from the associated images and sewer inspection metadata. Para[0049] & Fig. 2 teaches the sewer specific characteristics of the sewer line could also include structural defects, such as, for example, cracks in the sewer line, fractures in the sewer line, holes in the sewer line, deformations in the sewer line, collapses of sections of the sewer line, etc. para[0045] teaches characteristic identification and categorization system can use a single general CNN with finely tuned hyperparameters to obtain the model performing the image analysis and predict (i.e. identify and categorize) the different sewer specific characteristics depicted in the images. Para[0062] identification module 44 is a characteristic CNN module using one or more CNNs 42 that are associated with the sewer specific characteristics to be inspected in the associated sewer line and use the models obtained from the selected CNNs 42 to identify and categorize one or more characteristics of the sewer line depicted in the images 27 (or predict the absence thereof). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin with the method for automated inspection of sewer line using machine learning of Michaud in order to provide a system for characteristic identification and categorization system process the sewer inspection data and proceed with the prediction of the presence of the sewer specific characteristics and categorization of the characteristics.
Currin in view of Michaud does not explicitly a first step to automatically detect the at least one anomaly with a machine learning protocol of an anomaly detection program in real-time in response to the at least one anomaly being viewed by on a live video stream recorded by an optical imaging device of the system at a single viewpoint . However Choi discloses a first step to automatically detect the at least one anomaly with a machine learning protocol of an anomaly detection program in real-time in response to the at least one anomaly being viewed by on a live video stream para[0018] teaches the method for detecting defects in water supply and sewage facilities according to various embodiments disclosed in this document and the defect detection system supporting the method detect structural defects inside water supply and sewerage facilities in real time from images taken of the inside of the water supply and sewage facilities, thereby reducing user dependence due to post-defect detection can be reduced. Para[0041] teaches the defect detection module 140 May determine whether a defect is included in the real-time image data obtained from the water and sewage facility. For example, when the defect detection module 140 receives real-time image data using an exploration device (e.g. an intelligent pig)Para[0048] teaches to operation 240, the defect detection system 100 May determine whether a defect is included in the real-time image data obtained from the water and sewage facility. Para[0062]-[0065] teaches to step S320, the water and sewage facility 101 May acquire real-time image data inside the facility. The real-time image data may be, for example, an image captured using an exploration device (e.g. an intelligent pig).Referring to step S321, the water and sewage facility 101 May transmit the real-time image data obtained in step S321 to the defect detection system 100.Referring to step S322, the defect detection system 100 May receive real-time image data from the water and sewage facility 101.Referring to operation S323, the defect detection system 100 May determine whether a defect is included in the real-time image data received in operation S322) recorded by an optical imaging device of the system at a single viewpoint para[0008] teaches , the image converter may generate the at least one predicted image data to include at least one region of interest (ROI).para[0038] teaches the image conversion unit 120 May include at least one region of interest (ROI) in the prediction image data. The at least one region of interest may include, for example, a specific region assuming that a crack has occurred in the pipe of the water and sewage facility); It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin in view of Michaud with the method possible to provide a water and sewage facility defect detection of Choi in order to provide a system capable of detecting structural defects inside the water and sewage facility in real time from an image of the inside of the water and sewage facility, and a defect detection supporting the same.
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached) and Gubbi Lakshminarasimha et al. (US 20220036541 A1)
Regarding claim 2, Currin in view of Michaud and Choi discloses the method of claim 1, Currin in view of Michaud and Choi does not explicitly disclose wherein the step of automatically detecting the at least one anomaly from the machine learning protocol further comprises: judging a video format of the video stream by a video quality analyzer of the machine learning protocol. However Gubbi Lakshminarasimha discloses wherein the step of automatically detecting the at least one anomaly from the machine learning protocol further comprises: judging a video format of the video stream by a video quality analyzer of the machine learning protocol (para[0081] teaches a sewer pipeline, wherein an image is captured and frames are pre-processed wherein quality of frames is analyzed and frames with best quality is passed to further processing, para[0085] teaches extraction of meta-data about the survey from the corresponding video and frame data, loading pre-trained Artificial Intelligence, Deep Learning/Machine Learning (DL/ML) models comprised in the memory 102, computer vision (CV) techniques for detecting anomalies corresponding to different use cases such as manhole, liquid/fluid (e.g., water as liquid example) level, junction/connection, cracks/fractures/breaks, roots/holes, attached/floating deposits, and the like. Model (DL/ML) training that involves extracting data from videos, segregating them into train and test sets, annotating them with the class labels from the log provided, training them to detect frame/object level defects, as the case may be and test them using the validation set). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin of Michaud and Choi with the method of artificial intelligence/machine learning models and image processing techniques to automate log and data processing, reports and insights generation thereby reduce dependency on manual analysis of Gubbi Lakshminarasimha in order to provide a system improve/enhance accuracy in defect identification, analysis, classification thereof.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached) in further view of Asmari et al. (US 2024/0408773 A1).
Regarding claim 4, Currin in view of Michaud and Choi discloses the method of claim 1, Currin in view of Michaud and Choi does not explicitly disclose wherein the step of automatically detecting the at least one anomaly from the machine learning protocol further comprises: determining the at least one anomaly is a cross-bore defined in the conduit by a cross-bore analyzer of the machine learning protocol. However Asmari discloses wherein the step of automatically detecting the at least one anomaly from the machine learning protocol further comprises: determining the at least one anomaly is a cross-bore defined in the conduit by a cross-bore analyzer of the machine learning protocol (Para[0040] teaches defects, anomalies, or similar of the underground pipe system 101. Para[0046] teaches a feature 425 can include a tap, a joint, a crack, an elbow, a tee, a cross-bore, corrosion, a valve, etc. The system 100 can generate an identifier 430 provided in the form of a feature label, or similar. In some embodiments, the identifier 430 can be provided in the form of a general classification of the feature (e.g., a tap, a joint, a crack, an elbow, a tee, a cross-bore, corrosion, a valve, etc.) & Para[0049] & Fig. 6 teaches the feature classification 615 can include a general classification for the type of feature (e.g., a tap, a joint, a crack, an elbow, a tee, a cross-bore, corrosion, a valve, etc.). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin in view of Michaud and Choi with the method of collaborative robotic system for automatically detecting, locating, and mapping the anomalies of underground assets of Asmari in order to provide a system that efficiently collect the data related to the internal features and defects of a pipe system without causing an interruption of utility service to users, and ensures cost-effective identification and accurate objective classification of anomalies within underground infrastructures.
Claims 8, 17 are rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached) in further view of Spivey et al. (US 2020/0051237 A1).
Regarding claim 8, Currin in view of Michaud and Choi discloses the method of claim 1, Currin in view of Michaud and Choi does not explicitly disclose further comprising: updating the machine learning protocol of the anomaly detection program by a universal serial bus (USB) repository component. However Spivey discloses further comprising: updating the machine learning protocol of the anomaly detection program by a universal serial bus (USB) repository component (Para[0109] teaches computing units 1702 can have a processor 1722 and associated local tangible, computer readable media, such as memory 1724 and storage 1726. The processor 1722 may include a single processing core, multiple processing cores, a GPU, or any combinations thereof. The memory 1724 may include ROM and/or RAM used to store code, for example, used to direct the processor 1722 to implement the methods illustrated in FIGS. 6A, 6B, and 8. The storage 1726 may include one or more hard drives, one or more optical drives, one or more flash drives, or any combinations thereof. The storage 1726 may be used to provide storage for intermediate results, data, images, or code associated with operations, including code used to implement the methods of FIGS. 6A, 6B, and 8. In some examples, each of the computing units 1702 can include a neural network circuit 1728 that can train neural networks to identify components in a drill bit or bottom hole assembly and identify damage to a component of a drill bit or bottom hole assembly as described above). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin in view of Michaud and Choi with the method involves identifying a location, an extent, a type, a consistency, or any combinations of damage to the bit or the bottom hole assembly from an image of the bit or the bottom hole assembly via a supervised learning model of Spivey in order to provide a Increases the accuracy of the second neural network by combining multiple images of various components of a drill string. Prevents undesirable deviations
Regarding claim 17, Currin in view of Michaud and Choi discloses the method of claim 10, Currin in view of Michaud and Choi does not explicitly disclose further comprising: a universal serial bus (USB) repository component operatively in communication with the controller and configured to provide at least one update for the machine learning protocol. However Spivey discloses further comprising: a universal serial bus (USB) repository component operatively in communication with the controller and configured to provide at least one update for the machine learning protocol (Para[0109] teaches computing units 1702 can have a processor 1722 and associated local tangible, computer readable media, such as memory 1724 and storage 1726. The processor 1722 may include a single processing core, multiple processing cores, a GPU, or any combinations thereof. The memory 1724 may include ROM and/or RAM used to store code, for example, used to direct the processor 1722 to implement the methods illustrated in FIGS. 6A, 6B, and 8. The storage 1726 may include one or more hard drives, one or more optical drives, one or more flash drives, or any combinations thereof. The storage 1726 may be used to provide storage for intermediate results, data, images, or code associated with operations, including code used to implement the methods of FIGS. 6A, 6B, and 8. In some examples, each of the computing units 1702 can include a neural network circuit 1728 that can train neural networks to identify components in a drill bit or bottom hole assembly and identify damage to a component of a drill bit or bottom hole assembly as described above.). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of sewer inspection for many municipalities is inserting video cameras down a manhole and running the cameras along the inside of the sewer main in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin in view of Michaud and Choi with the method involves identifying a location, an extent, a type, a consistency, or any combinations of damage to the bit or the bottom hole assembly from an image of the bit or the bottom hole assembly via a supervised learning model of Spivey in order to provide a Increases the accuracy of the second neural network by combining multiple images of various components of a drill string. Prevents undesirable deviations.
Claims 9, 18 are rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached) in further view of Moloney et al. (WO2024061986 A1)
Regarding claim 9, Currin in view of Michaud and Choi discloses the method of claim 1, Currin in view of Michaud and Choi does not explicitly disclose wherein the step of indicating the at least one detected anomaly on the live video stream further includes an alert system that is accessible by the controller. However Moloney discloses wherein the step of indicating the at least one detected anomaly on the live video stream further includes an alert system that is accessible by the controller (Para[00111] teaches anomaly detection unit 110c sends a notification or alert to an operator apparatus or console for alerting an operator monitoring the wastewater network 102 of the anomaly, para[0349] teaches Once detected, the anomaly detection unit 110c may send an alert indicating that an upstream blockage anomaly has occurred at wastewater asset 104i with pump system 109i. Para[0073] teaches real-time detection of wastewater asset anomalies such as, without limitation, for example wastewater pump malfunctions of a pump system, upstream or downstream blockages and/or sensor issues associated with wastewater network 102. Para[0094] teaches if an anomaly is detected based on either first and/or second pump ML models, then an operator of the wastewater network 102 may be alerted). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin in view of Michaud and Choi with the method in which anomaly is detected based on ML models, then an operator of the wastewater network may be alerted in order to provide a system that takes the advantage of early scheduling and deployment of maintenance personnel for restoring or returning said wastewater asset with wastewater pump system back to a normal behavior.
Regarding claim 18, Currin in view of Michaud and Choi discloses the method of claim 10, Currin in view of Michaud and Choi does not expclity disclose further comprising: an alert system that is accessible by the controller for applying alerts on the at least one anomaly detected. However Moloney discloses further comprising: an alert system that is accessible by the controller for applying alerts on the at least one anomaly detected Para[00111] teaches anomaly detection unit 110c sends a notification or alert to an operator apparatus or console for alerting an operator monitoring the wastewater network 102 of the anomaly, para[0349] teaches Once detected, the anomaly detection unit 110c may send an alert indicating that an upstream blockage anomaly has occurred at wastewater asset 104i with pump system 109i. Para[0073] teaches real-time detection of wastewater asset anomalies such as, without limitation, for example wastewater pump malfunctions of a pump system, upstream or downstream blockages and/or sensor issues associated with wastewater network 102. Para[0094] teaches if an anomaly is detected based on either first and/or second pump ML models, then an operator of the wastewater network 102 may be alerted). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin in view of Michaud and Choi with the method in which anomaly is detected based on ML models, then an operator of the wastewater network may be alerted in order to provide a system that takes the advantage of early scheduling and deployment of maintenance personnel for restoring or returning said wastewater asset with wastewater pump system back to a normal behavior .
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached) in further view of Lei et al. (CN117911741 A) (machine translation attached)
Regarding claim 11, Currin in view of Michaud and Choi discloses the system of claim 10, Currin in view of Michaud and Choi does not explicitly disclose, wherein the machine learning protocol comprises: a video quality analyzer operatively in communication with a video transcoding architecture of the anomaly detection program. However Lei discloses wherein the machine learning protocol comprises: a video quality analyzer operatively in communication with a video transcoding architecture of the anomaly detection program (Para[0036] teaches gas pipeline illegal occupation detection method based on machine learning provided by the present invention provides high-quality input for subsequent machine learning models by establishing and preprocessing a pipeline image data set, thereby improving the accuracy of model training; Para[0056] teaches by performing color space conversion on the pipeline image data set and extracting statistical features, the color space conversion includes performing an XYZ transformation on the training set images, transforming them from the RGB color space to the XYZ color space). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin of Michaud and Choi with the method of image analysis module is used to analyze the pipeline image to be identified through the trained image SVM classifier and determine the occupancy behavior of the pipeline of Lei in order to provide a system detecting illegal encroachment on pipelines.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached) in further view of Scharpf et al. (US 2023/0186410 A1).
Regarding claim 13, Currin in view of Michaud and Choi discloses the method of claim 1, Currin in view of Michaud and Choi does not explicitly disclose the system of claim 10, wherein the machine learning protocol further comprises: a cross-bore analyzer operatively in communication with a video transcoding architecture of the anomaly detection program and configured with cross-bore assessment guidelines. However Scharpf discloses wherein the machine learning protocol further comprises: a cross-bore analyzer operatively in communication with a video transcoding architecture of the anomaly detection program and configured with cross-bore assessment guidelines (Abstract teaches cross-bore risk management involves receiving at least one dataset comprising a plurality of assets and cross-bore data. A risk probability value is calculated, using a processor, based on the cross-bore data for each asset of the plurality of assets using machine learning techniques). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin of Michaud and Choi with the prevention procedure to be modified is effected to reduce risk of generating future cross bores based on a machine learning technique that has evaluated the risk probability values and the presence of the cross bore is validated in order to provide a system in which risk probability for a specified geographical area based on the spatially distributed risk probability values, ensuring that the cross-bore risk management is efficiently performed.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached) in further view of Gubbi Lakshminarasimha et al. (US 20220036541A1) and Asmari et al. (US 2024/0408773 A1).
Regarding claim 20, Currin in view of Michaud and Choi in further view of Gubbi Lakshminarasimha discloses the computer program product of claim 19, Currin in view of Michaud further discloses wherein the step of executing by the controller the first step to automatically detect the at least one anomaly with the machine learning protocol further comprising: executing, by the controller, a fifth step to determine a type of anomaly of the at least one anomaly by a conduit assessor of the machine learning protocol (Michaud: Para[0040] teaches sewer specific characteristics of the sewer line can include attributes (or observation types) which are relevant to the description, condition and/or performance of the sewer line. the sewer specific characteristics of the sewer line can include attributes described in the Pipeline Assessment Certification Program® (PACP®) from NASSCO.. The sewer specific characteristics of the sewer line could also include structural defects, such as, for example, cracks in the sewer line, fractures in the sewer line, holes in the sewer line, deformations in the sewer line, collapses of sections of the sewer line, etc.), wherein the conduit assessor is loaded with pipe, lateral, and manhole assessment coding guidelines (Currin: para[0079] teaches Each video segment may also be associated with high-level information concerning when it was captured, an overall five star rating (e.g., a Pipeline Assessment and Certification Program (PACP) rating) of the pipe segment shown in or otherwise corresponding to the video segment & Para[0141] teaches PACP data pane 1312 comprises PACP information, such as structural ratings, operations and maintenance ratings, and overall ratings. PACP-certified inspections allow for software to issue these ratings on the inspected pipe sections. Ratings pane 1313 comprises a star rating, which may be initially based on the overall index shown in PACP data pane 1312. & Para[0163] teaches PACP, but also with the metadata requirements for inspections under the Lateral Assessment and Certification Program (LACP) and Manhole Assessment and Certification Program (MACP)).
Currin in view of Michaud and Choi does not explicitly disclose executing, by the controller, a fourth step to judge the live video stream by a video quality analyzer of the machine learning protocol; and executing, by the controller, a sixth step to determine the at least one anomaly is a cross-bore defined in the conduit by a cross-bore analyzer of the machine learning protocol. However Gubbi Lakshminarasimha discloses executing, by the controller, a fourth step to judge the live video stream by a video quality analyzer of the machine learning protocol para[0081] teaches a sewer pipeline, wherein an image is captured and frames are pre-processed wherein quality of frames is analyzed and frames with best quality is passed to further processing, para[0085] teaches extraction of meta-data about the survey from the corresponding video and frame data, loading pre-trained Artificial Intelligence, Deep Learning/Machine Learning (DL/ML) models comprised in the memory 102, computer vision (CV) techniques for detecting anomalies corresponding to different use cases such as manhole, liquid/fluid (e.g., water as liquid example) level, junction/connection, cracks/fractures/breaks, roots/holes, attached/floating deposits, and the like. Model (DL/ML) training that involves extracting data from videos, segregating them into train and test sets, annotating them with the class labels from the log provided, training them to detect frame/object level defects, as the case may be and test them using the validation set). It would have been obvious to one having ordinary skill in e art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin and Michaud and Choi with the method of artificial intelligence/machine learning models and image processing techniques to automate log and data processing, reports and insights generation thereby reduce dependency on manual analysis of Gubbi Lakshminarasimha in order to provide a system improve/enhance accuracy in defect identification, analysis, classification thereof.
Currin in view of Michaud and Choi in further view of Gubbi Lakshminarasimha does not explicitly disclose and executing, by the controller, a sixth step to determine the at least one anomaly is a cross-bore defined in the conduit by a cross-bore analyzer of the machine learning protocol. However Asmari discloses and executing, by the controller, a sixth step to determine the at least one anomaly is a cross-bore defined in the conduit by a cross-bore analyzer of the machine learning protocol (Para[0040] teaches defects, anomalies, or similar of the underground pipe system 101. Para[0045] -[0046] teaches a “advanced training model” can include machine learning processes, artificial intelligence processes, and other similar advanced machine learning processes. For example, the system and processes of the present disclosure can provide an objective classification of one or more features 425 of a pipe asset having different individualized parameters. The system can leverage the known characteristics of features with similar metrics as an input to an iterative training process for an automated detection and objective classification of a feature 425 based one or more parameters. Feature 425 can include a tap, a joint, a crack, an elbow, a tee, a cross-bore, corrosion, a valve, etc. The system 100 can generate an identifier 430 provided in the form of a feature label, or similar. In some embodiments, the identifier 430 can be provided in the form of a general classification of the feature (e.g., a tap, a joint, a crack, an elbow, a tee, a cross-bore, corrosion, a valve, etc.) & Para[0049] & Fig. 6 teaches the feature classification 615 can include a general classification for the type of feature (e.g., a tap, a joint, a crack, an elbow, a tee, a cross-bore, corrosion, a valve, etc.). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of by obtaining frames with best quality of Currin in view of Michaud and Choi in further view of Gubbi Lakshminarasimha with the method of Collaborative robotic system for automatically detecting, locating, and mapping the anomalies of underground assets of Asmari in order to provide a system that efficiently collect the data related to the internal features and defects of a pipe system without causing an interruption of utility service to users, and ensures cost-effective identification and accurate objective classification of anomalies within underground infrastructures.
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Currin et al. (US 2015/0310360 A1) in view of Michaud et al. (US 2021/0181119 A1) and Choi et al. (WO 2023182855 A1) (machine translation attached) in further view of i Kishi et al. (WO 2021131998A1 ) (machine translation attached).
Regarding claim 21, Currin in view of Michaud and Choi discloses the method of claim 1, Currin in view of Michaud and Choi does not explicitly disclose wherein the step of automatically detecting the at least one anomaly from the machine learning protocol further comprises: setting a start time delay and an end time delay, by a delay adjuster of the machine learning protocol, when the optical timing device is outside of the conduit. However Kishi discloses wherein the step of automatically detecting the at least one anomaly from the machine learning protocol further comprises: setting a start time delay and an end time delay, by a delay adjuster of the machine learning protocol, when the optical timing device is outside of the conduit (para[0016] teaches the delay information addition unit 11 adds information related to the delay of the current work state to the previous notification content. Para[0076]-[0077] & FIG. 26 is an image diagram illustrating an example of a captured image 80 in a pipe transport operation according to a shield inner pipe (including a joint groove inner pipe. When the actual work time corresponding to the conveyance work of the pipe is longer than the upper limit value, the delay information addition unit 11 adds information related to the delay of the conveyance work of the pipe to the previous notification content. Examples of factors that increase the work time in the transportation work of the pipe include the occurrence of some trouble in the middle of transportation. Therefore, as an example of the information related to the delay, the notification device 102 May include information indicating that "there is a possibility that a trouble has occurred in the pipe transport operation"). It would have been obvious to one having ordinary skill in the art before the effective filing date of the invention to use the method of automated inspection of sewer line using machine learning in order to capture video of the walls of the sewer main effective at spotting cracks in the walls, bad joints, root growth, FOG, or other faults of Currin of Michaud and Choi with mage detection unit that detects a feature image from a captured image including a work target area and adds information related to the delay of the current work state to the previous notification content of Kishi in order to provide a system in which , actions that add information related to delays in the current work status to the notification content were performed during excavation (sheet pile installation) work, pipe lowering and installation work, and joint splicing and inspection work.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ROWINA J CATTUNGAL whose telephone number is (571)270-5922. The examiner can normally be reached Monday-Thursday 7:30-6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Brian Pendleton can be reached at (571) 272-7527. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ROWINA J CATTUNGAL/Primary Examiner, Art Unit 2425
Prosecution Insights