DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim Rejections - 35 USC § 103
Claim 1 is rejected under 35 U.S.C. 103 as being unpatentable over Han (CN117392401 A), in view of Ling (Data Modeling Techniques for Pipeline Integrity Assessment: A State-of-the-Art Survey, 2023).
Regarding Claim 1, Han teaches a method comprising:
receiving first data collected as a first pipeline inspection gauge (PIG) passes through a first pipeline (Summary of the Invention, Page 1: “In one aspect, the present disclosure provides a method for monitoring travel of a pipe cleaning detection device, including: continuously collecting vibration signal data in the pipeline through the Φ-OTDR sensor installed in the pipeline”. Notes: a pip cleaning detection device reads on a pipeline inspection gauge, as pipeline inspection gauges are used for cleaning and detecting anomalies. Φ-OTDR sensors are used for recording fiber strain or motion);
receiving second data collected via an optical fiber as a second PIG passes through a second pipeline, wherein the second data is representative of one or more fiber optic events (Summary of the Invention, Page 1: “In one aspect, the present disclosure provides a method for monitoring travel of a pipe cleaning detection device, including: continuously collecting vibration signal data in the pipeline through the Φ-OTDR sensor installed in the pipeline”. Notes: a pip cleaning detection device reads on a pipeline inspection gauge, as pipeline inspection gauges are used for cleaning and detecting anomalies. Φ-OTDR sensors are used for recording optic fiber strain or motion. Fiber optic events, in the broadest reasonable interpretation, are any events of note regarding fiber optics, which includes the vibration or strain of fiber optics corresponding to PIG runs);
applying the trained machine learning model to the second data (Summary of the Invention, Page 1: “converting the vibration feature into an image feature by using an image recognition technology, and performing deep learning on the image feature in combination with the 3D multi-convolutional neural network technology to determine a traveling position of the cleaning tube detection device”);
identifying a position of the second PIG (Summary of the Invention, Page 1: “converting the vibration feature into an image feature by using an image recognition technology, and performing deep learning on the image feature in combination with the 3D multi-convolutional neural network technology to determine a traveling position of the cleaning tube detection device”); and
generating a graphical user interface (GUI) to display the position of the second PIG (Detailed Description of the Embodiments, Page 5: “the platform provides an operation interface for monitoring the state of travel of the pig and providing early warning information, data viewing and alarm functions”. Notes: the operation interface enables data viewing, and hence qualifies as a GUI due to its visual interface).
Han does not explicitly teach training a machine learning model using the first data, although implicitly, a model is trained on data of the same or similar type as the data that it evaluates.
However, Ling teaches training a machine learning model using the first data (Abstract: “First, the description of data required to construct pipeline defect integrity assessment models is presented, where the required data for modeling can be obtained from pipeline inspection measurements, monitoring sensors, testing experiments, etc; Section I, Introduction; “Data-driven models are usually combined with machine learning techniques” Section II A, Data for Defect Characterization: “Pipeline ILI uses PIG that moves along the pipeline and is carried out periodically to detect any pipeline defects [31]. It is a direct and reliable way to collect the signals with defect information for defect characterization”).
Han and Ling are considered analogous in the art with respect to obtaining data via PIG runs, and subsequently using machine learning models with the obtained data. Machine learning models can be trained on data obtained via PIG runs, as is evident in Ling; one would be motivated to do so to develop their own model trained on accessible data.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the use of a machine learning model with data obtained via PIG runs for determining a PIG position of Han with the training of a machine learning model on data obtained via PIG runs of Ling; Doing so would yield the predictable result of having a machine learning model trained on accessible PIG run data for determining a PIG position.
Claims 9 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Han (CN117392401 A), in view of Ling (Data Modeling Techniques for Pipeline Integrity Assessment: A State-of-the-Art Survey, 2023) and Stajanca (Detection of Leak-Induced Pipeline Vibrations Using Fiber—Optic Distributed Acoustic Sensing, 2018).
Regarding Claim 9, Han teaches A system, comprising:
processing circuitry; and a memory, accessible by the processing circuitry, and storing instructions that, when executed by the processing circuitry, cause the processing circuitry to perform operations (Detailed Description of the Embodiments, Page 6: “Further, the present disclosure further provides an electronic device, which may include a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory complete communication with each other through the communication bus. The processor may invoke the computer program in the memory to perform the travel monitoring method for the pipe cleaning detection device”) comprising: receiving first data collected as a first pipeline inspection gauge (PIG) passes through a first pipeline (Summary of the Invention, Page 1: “In one aspect, the present disclosure provides a method for monitoring travel of a pipe cleaning detection device, including: continuously collecting vibration signal data in the pipeline through the Φ-OTDR sensor installed in the pipeline”. Notes: a pip cleaning detection device reads on a pipeline inspection gauge, as pipeline inspection gauges are used for cleaning and detecting anomalies. Φ-OTDR sensors are used for recording fiber strain or motion);
receiving second data collected via an optical fiber as a second PIG passes through a second pipeline, wherein the second data is representative of one or more fiber optic events (Summary of the Invention, Page 1: “In one aspect, the present disclosure provides a method for monitoring travel of a pipe cleaning detection device, including: continuously collecting vibration signal data in the pipeline through the Φ-OTDR sensor installed in the pipeline”. Notes: a pip cleaning detection device reads on a pipeline inspection gauge, as pipeline inspection gauges are used for cleaning and detecting anomalies. Φ-OTDR sensors are used for recording fiber strain or motion. Fiber optic events, in the broadest reasonable interpretation, are any events of note regarding fiber optics, which includes the vibration or strain of fiber optics corresponding to PIG runs);
applying the trained machine learning model to the second data (Summary of the Invention, Page 1: “converting the vibration feature into an image feature by using an image recognition technology, and performing deep learning on the image feature in combination with the 3D multi-convolutional neural network technology to determine a traveling position of the cleaning tube detection device”);
identifying a position of the second PIG (Summary of the Invention, Page 1: “converting the vibration feature into an image feature by using an image recognition technology, and performing deep learning on the image feature in combination with the 3D multi-convolutional neural network technology to determine a traveling position of the cleaning tube detection device”); and
generating a graphical user interface (GUI) to display the position of the second PIG (Detailed Description of the Embodiments, Page 5: “the platform provides an operation interface for monitoring the state of travel of the pig and providing early warning information, data viewing and alarm functions”. Notes: the operation interface enables data viewing, and hence qualifies as a GUI due to its visual interface).
Han does not explicitly teach training a machine learning model using the first data, although implicitly, a model is trained on data of the same or similar type as the data that it evaluates.
However, Ling teaches training a machine learning model using the first data (Abstract: “First, the description of data required to construct pipeline defect integrity assessment models is presented, where the required data for modeling can be obtained from pipeline inspection measurements, monitoring sensors, testing experiments, etc; Section I, Introduction; “Data-driven models are usually combined with machine learning techniques” Section II A, Data for Defect Characterization: “Pipeline ILI uses PIG that moves along the pipeline and is carried out periodically to detect any pipeline defects [31]. It is a direct and reliable way to collect the signals with defect information for defect characterization”).
Han and Ling are considered analogous in the art with respect to obtaining data via PIG runs, and subsequently using machine learning models with the obtained data. Machine learning models can be trained on data obtained via PIG runs, as is evident in Ling; one would be motivated to do so to develop their own model trained on accessible data.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the use of a machine learning model with data obtained via PIG runs for determining a PIG position of Han with the training of a machine learning model on data obtained via PIG runs of Ling; Doing so would yield the predictable result of having a machine learning model trained on accessible PIG run data for determining a PIG position.
Han as modified does not teach automatically calibrating a distance along the optical fiber to distance along the second pipeline.
However, Stajanca teaches automatically calibrating a distance along an optical fiber to distance along a pipeline (Section 2.2, Employed Sensors: “Commercial DAS system (Helios DAS, Fotech Solutions, Church Crookham, UK) is employed in the study for the distributed vibration measurement. The system represents the simplest amplitude-based implementation of the DAS technology. As discussed earlier, designation DAS is, in fact, a misnomer for this type of system as it offers only qualitative distributed vibration measurement. Nevertheless, we will continue to use this nomenclature in the paper as it seems to be established in the field and the used system is sold under this name. All presented measurements were performed using 200 ns laser pulse length and 80 kHz pulse repetition rate. Relatively large pulse length helps to increase system sensitivity for potential weak leak-induced signals. On the other hand, long pulse length 𝑇𝑝 limits the achievable DAS spatial resolution 𝑟≅𝑐𝑇𝑝/(2𝑛𝑒𝑓𝑓), where 𝑐 is the speed of light in vacuum and 𝑛𝑒𝑓𝑓 is the effective refractive index of the optical fiber mode. Pulse length of 200 ns corresponds to roughly 20 m spatial resolution in the fiber. The lower DAS spatial resolution using long pulse lengths is compensated by fiber application with high fiber-to-pipe coverage ratio. For our fiber application, 200 ns pulse in the helically applied fiber corresponds to measurement spatial resolution of roughly 1.6 m long pipe section. This might not be sufficient to perform precise signal localization within the same pipe segment, however, localization of the signals between the different pipe segments should be possible”. Notes: automatically calibrating a distance along an optic fiber to distance along a pipeline, in its broadest reasonable interpretation, is taken to mean any calibration process where a distance along a pipeline is adjusted according to a fiber optic distance via a mechanical process. As noted by Stajanca, the relationship between pulse length and fiber optic distance is defined, where the fiber optic distance is proportional to the pipe distance. Considering that DAS technology can adjust pulse length through an automated process, the utilization of DAS technology enables the automatic calibration of distance along an optic fiber to distance along a pipeline via controlling pulse length).
Han as modified and Stajanca are considered analogous in the art with respect to the use of distributed acoustic sensing (DAS) for determining positions in pipelines. A common motivation is to utilize a DAS system configured for fiber optic distance length corresponding with a segment of pipeline, as is evident in Stajanca. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that DAS pulse length can be adjusted in a DAS system, and hence result in the automatic calibration along a distance along an optical fiber to distance along a pipeline, as is specified in Stajanca.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the system determining a PIG position within a pipeline using machine learning of Han as modified with the automatic calibration of a distance along an optical fiber to a distance along a pipeline of Stajanca; Doing so would yield the predictable result of customizing the system to the required specifications of the pipeline, DAS system, and fiber optics.
Regarding Claim 16, Han teaches a non-transitory, computer readable medium comprising instructions that, when executed by a processing circuitry, cause the processing circuitry to perform operations (Detailed Description of the Embodiments, Page 6: “Further, the present disclosure further provides an electronic device, which may include a processor, a communication interface, a memory, and a communication bus. The processor, the communication interface, and the memory complete communication with each other through the communication bus. The processor may invoke the computer program in the memory to perform the travel monitoring method for the pipe cleaning detection device”; Summary of the Invention, page 2: “Optionally, the memory is a non-transitory computer-readable storage medium”) comprising:
receiving first data collected as a first pipeline inspection gauge (PIG) passes through a first pipeline (Summary of the Invention, Page 1: “In one aspect, the present disclosure provides a method for monitoring travel of a pipe cleaning detection device, including: continuously collecting vibration signal data in the pipeline through the Φ-OTDR sensor installed in the pipeline”. Notes: a pip cleaning detection device reads on a pipeline inspection gauge, as pipeline inspection gauges are used for cleaning and detecting anomalies. Φ-OTDR sensors are used for recording fiber strain or motion);
receiving second data collected via an optical fiber as a second PIG passes through a second pipeline, wherein the second data is representative of one or more fiber optic events (Summary of the Invention, Page 1: “In one aspect, the present disclosure provides a method for monitoring travel of a pipe cleaning detection device, including: continuously collecting vibration signal data in the pipeline through the Φ-OTDR sensor installed in the pipeline”. Notes: a pip cleaning detection device reads on a pipeline inspection gauge, as pipeline inspection gauges are used for cleaning and detecting anomalies. Φ-OTDR sensors are used for recording fiber strain or motion. Fiber optic events, in the broadest reasonable interpretation, are any events of note regarding fiber optics, which includes the vibration or strain of fiber optics corresponding to PIG runs);
applying the trained machine learning model to the second data (Summary of the Invention, Page 1: “converting the vibration feature into an image feature by using an image recognition technology, and performing deep learning on the image feature in combination with the 3D multi-convolutional neural network technology to determine a traveling position of the cleaning tube detection device”);
identifying a position of the second PIG (Summary of the Invention, Page 1: “converting the vibration feature into an image feature by using an image recognition technology, and performing deep learning on the image feature in combination with the 3D multi-convolutional neural network technology to determine a traveling position of the cleaning tube detection device”); and
generating a graphical user interface (GUI) to display the position of the second PIG (Detailed Description of the Embodiments, Page 5: “the platform provides an operation interface for monitoring the state of travel of the pig and providing early warning information, data viewing and alarm functions”. Notes: the operation interface enables data viewing, and hence qualifies as a GUI due to its visual interface),
overlaying the second data with the GUI (Detailed Description of the Embodiments, Page 5: “the platform provides an operation interface for monitoring the state of travel of the pig and providing early warning information, data viewing and alarm functions”. Notes: the operation interface enables data viewing, and hence qualifies as a GUI due to its visual interface).
Han does not explicitly teach training a machine learning model using the first data, although implicitly, a model is trained on data of the same or similar type as the data that it evaluates.
However, Ling teaches training a machine learning model using the first data (Abstract: “First, the description of data required to construct pipeline defect integrity assessment models is presented, where the required data for modeling can be obtained from pipeline inspection measurements, monitoring sensors, testing experiments, etc; Section I, Introduction; “Data-driven models are usually combined with machine learning techniques” Section II A, Data for Defect Characterization: “Pipeline ILI uses PIG that moves along the pipeline and is carried out periodically to detect any pipeline defects [31]. It is a direct and reliable way to collect the signals with defect information for defect characterization”).
Han and Ling are considered analogous in the art with respect to obtaining data via PIG runs, and subsequently using machine learning models with the obtained data. Machine learning models can be trained on data obtained via PIG runs, as is evident in Ling; one would be motivated to do so to develop their own model trained on accessible data.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the use of a machine learning model with data obtained via PIG runs for determining a PIG position of Han with the training of a machine learning model on data obtained via PIG runs of Ling; Doing so would yield the predictable result of having a machine learning model trained on accessible PIG run data for determining a PIG position.
Han as modified does not teach automatically calibrating a distance along the optical fiber to distance along the second pipeline.
However, Stajanca teaches automatically calibrating a distance along an optical fiber to distance along a pipeline (Section 2.2, Employed Sensors: “Commercial DAS system (Helios DAS, Fotech Solutions, Church Crookham, UK) is employed in the study for the distributed vibration measurement. The system represents the simplest amplitude-based implementation of the DAS technology. As discussed earlier, designation DAS is, in fact, a misnomer for this type of system as it offers only qualitative distributed vibration measurement. Nevertheless, we will continue to use this nomenclature in the paper as it seems to be established in the field and the used system is sold under this name. All presented measurements were performed using 200 ns laser pulse length and 80 kHz pulse repetition rate. Relatively large pulse length helps to increase system sensitivity for potential weak leak-induced signals. On the other hand, long pulse length 𝑇𝑝 limits the achievable DAS spatial resolution 𝑟≅𝑐𝑇𝑝/(2𝑛𝑒𝑓𝑓), where 𝑐 is the speed of light in vacuum and 𝑛𝑒𝑓𝑓 is the effective refractive index of the optical fiber mode. Pulse length of 200 ns corresponds to roughly 20 m spatial resolution in the fiber. The lower DAS spatial resolution using long pulse lengths is compensated by fiber application with high fiber-to-pipe coverage ratio. For our fiber application, 200 ns pulse in the helically applied fiber corresponds to measurement spatial resolution of roughly 1.6 m long pipe section. This might not be sufficient to perform precise signal localization within the same pipe segment, however, localization of the signals between the different pipe segments should be possible”. Notes: automatically calibrating a distance along an optic fiber to distance along a pipeline, in its broadest reasonable interpretation, is taken to mean any calibration process where a distance along a pipeline is adjusted according to a fiber optic distance via a mechanical process. As noted by Stajanca, the relationship between pulse length and fiber optic distance is defined, where the fiber optic distance is proportional to the pipe distance. Considering that DAS technology can adjust pulse length through an automated process, the utilization of DAS technology enables the automatic calibration of distance along an optic fiber to distance along a pipeline via controlling pulse length).
Han as modified and Stajanca are considered analogous in the art with respect to the use of distributed acoustic sensing (DAS) for determining positions in pipelines. A common motivation is to utilize a DAS system configured for fiber optic distance length corresponding with a segment of pipeline, as is evident in Stajanca. It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that DAS pulse length can be adjusted in a DAS system, and hence result in the automatic calibration along a distance along an optical fiber to distance along a pipeline, as is specified in Stajanca.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the system determining a PIG position within a pipeline using machine learning of Han as modified with the automatic calibration of a distance along an optical fiber to a distance along a pipeline of Stajanca; Doing so would yield the predictable result of customizing the system to the required specifications of the pipeline, DAS system, and fiber optics.
Claims 2-3 are rejected under 35 U.S.C. 103 as being unpatentable over Han (CN117392401 A), in view of Ling (Data Modeling Techniques for Pipeline Integrity Assessment: A State-of-the-Art Survey, 2023), and in further view of Ozharar (US 20230130788 A1), Hill (US 8973444 B2) and Munoz (Finding Well-Coupled Optical Fiber Locations for Railway Monitoring Using Distributed Acoustic Sensing, 2023, 2023).
Regarding Claim 2, the method of Claim 1 is rejected over Han as modified.
Han as modified teaches data comprising a waterfall image for training the machine learning model (Han, Summary of the Invention: “representing the vibration intensity of each region with different colors, and then drawing the vibration intensity of each region on the whole optical cable according to time and space into a waterfall diagram; The vibration intensity of different color depths on the waterfall image is taken as the image feature, and deep learning is performed by using the 3D multi-convolutional neural network to determine the traveling position of the cleaning tube detection device”), and
Providing the training images to the machine learning model during a training cycle (Han, Summary of the Invention: “representing the vibration intensity of each region with different colors, and then drawing the vibration intensity of each region on the whole optical cable according to time and space into a waterfall diagram; The vibration intensity of different color depths on the waterfall image is taken as the image feature, and deep learning is performed by using the 3D multi-convolutional neural network to determine the traveling position of the cleaning tube detection device”. Notes: because deep learning is performed using the waterfall image (training image), using a neural network, a training cycle is necessarily performed because learning is performed).
Han as modified does not teach splitting the waterfall images into a plurality of sub-images, and does not teach generating a plurality of respective training images.
However, Ozharar teaches splitting the waterfall images into a plurality of sub-images, and generating a plurality of respective training images (Paragraph [0096]: “In the inference phase, the input waterfall images are converted into local patches with overlaps by sliding window. The patches are then classified by the trained TRN model to determine if an open or close event exists within the patch. The AI engine takes continuous streams of input data and runs inference in real-time on GPU, providing timestamps, event types, and confidence scores as the output”).
Han as modified and Ozharar are considered analogous in the art with respect to the use of waterfall images obtained from fiber optics for machine learning. One would be motivated to split a waterfall image into multiple sub images for training purposes by identifying whether certain fiber optic events occur in a particular patch, as is evident in Ozharar.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the machine learning model using waterfall images of Han as modified with the splitting of waterfall images into a plurality of sub-images for generating training images of Ozharar; Doing so would yield the predictable result of producing more specific training images isolating events of interest, improving the performance of the machine learning model.
Han as modified does not explicitly teach that each training image comprises a V-shaped plot.
However, Hill teaches establishing a window such that each training image comprises a V-shaped plot (Column 12, lines 60-67: “Two main features can be seen from the waterfall plot. The first is an area of constant acoustic disturbance towards the left of the plot at 402, corresponding to a length of approximately 4000 m of the sensing fibre. This is attributable to an industrial unit located over that section of fibre, producing a steady vibrational noise. Secondly distinct chevron (i.e. V shaped) patterns can be seen, most clearly in region 404, away from the constant noise of the industrial unit”; Column 13, lines 1-12: “The vertex of each chevron is located at point 406 along the fibre, corresponding to the origin of the pressure pulse. The `V` shape of the plot corresponds to the pressure pulse moving along the pipe in both directions away from the source of the pulse, and the slope of the `V` shape corresponds to the speed of sound in the pressurised fluid contained within the pipe which in this case is approximately 400 ms.sup.-1. It can be seen that a series of pressure pulses are introduced into the fluid, in this instance pressurized gas, and multiple traces are formed. On the top histogram plot, the individual pulses appear in their respective positions at that instant, spaced along the fibre”; Column 13, lines 37-39: “To simplify the detection a window of data is chosen such that only one pressure pulse is expected during the window”)
Han as modified and Hill are considered analogous in the art with respect to the derivation of PIG position via waterfall images. A common motivation in the art is to isolate the data into specific windows depicting the position of a PIG at an instance of time, as is evident in Hill.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the machine learning model using waterfall images for determining PIG positions of Han as modified with the isolation of waterfall images and their pulses on waterfall images into windows where only a single pulse (V-shaped chevron) is present; Doing so would result in the predictable result of isolating instances in time indicating PIG position in waterfall images, enabling a machine learning model to learn off of specific instances of data, improving the learning process.
Han as modified does not teach that training the machine learning model comprises using bounding boxes centered on the V-shaped plot for training.
However, Munoz teaches data comprising a waterfall image (Figure 2), and using bounding boxes centered on the V-shaped plot for identifying a position of interest (Figure 2).
Han as modified and Munoz are considered analogous in the art with respect to the use of data obtained via fiber optics represented as waterfall images. While Munoz does not explicitly teach using the bounding boxes centered on the V-shaped plot for training a machine learning model, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that training a machine learning model off images comprises using bounding boxes. Doing so is well established in the art as identifying areas of interest for identification by the model. Considering that Munoz identifies the V-shaped plot within the bounding box as being an area of interest with regards to the position of traveling entities, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that training a model to identify a position of a tracked entity (such as in Han as modified, which itself utilizes waterfall images) would utilize machine learning techniques utilizing waterfall images (such as that of Munoz, which identifies key positions of interest regarding position of entities in the waterfall image).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the training of a machine learning model using a waterfall image of Han as modified with the use of bounding boxes centered on the V-shaped plot for training the machine learning model of Munoz; Doing so would yield the predictable result of training a machine learning model through the established use of images annotated with bounding boxes for identifying regions of interest.
Regarding Claim 3, the method of Claim 2 is rejected over Han as modified.
Han as modified teaches that each of the sub-images comprises a plot of the first data over a time interval (Han, Summary of the Invention, page 2: “normalizing the four vibration feature values of each region to characterize the vibration intensity of each region; representing the vibration intensity of each region with different colors, and then drawing the vibration intensity of each region on the whole optical cable according to time and space into a waterfall diagram”; Munoz, Figure 2, while not strictly over time (time is on the y axis), can clearly be rearranged to be over time).
Claims 4-5 are rejected under 35 U.S.C. 103 as being unpatentable over Han (CN117392401 A), in view of Ling (Data Modeling Techniques for Pipeline Integrity Assessment: A State-of-the-Art Survey, 2023), Ozharar (US 20230130788 A1), Hill (US 8973444 B2) and Munoz (Finding Well-Coupled Optical Fiber Locations for Railway Monitoring Using Distributed Acoustic Sensing, 2023, 2023), and in further view of Stack Overflow A (How to fit/clip an axis aligned bounding box around a portion of a triangle, 2023).
Regarding Claim 4, the method of Claim 2 is rejected over Han as modified.
Han as modified teaches a bounding box corresponding to a location of the PIG (Hill, Column 13, lines 13-20: “It can be seen therefore that a pressure pulse is clearly detectable in the pipeline using a distributed acoustic sensor. As movement of a pig in the pipeline will generate a repetitive series of pressure pulses such a repetitive series can be used to locate the pig within the pipeline. Preferably however the characteristic V shape, caused by propagation of the pressure pulse in the both directions within the pipeline, may be used as an acoustic signature of the pig”).
Han as modified does not explicitly teach that the first and second corners of each respective bbox intersects with the V-shaped plot, and does not explicitly teach that a lower midpoint of an edge of each respective bbox opposite the first and second corners, corresponds to a location of the second PIG.
However, Stack Overflow A teaches fitting a bounding box to points defining a region of interest, such that the first and second corners of a bounding box intersects with a V-shape (Post by yosmo78: “I want to clip an axis aligned bounding box against a triangle in a way that creates a new tight fitting axis aligned bounding box around or in the triangle”; Refer to images in the post, which demonstrate aligning a bounding box to defined points; Answer by user1196549: “To get a correct result, you need to compute the intersection of the two shapes, for instance using the Sutherland-Hodgman algorithm, that you can specialize for the case of a triangle and a rectangle. If I am right, in the worst case you get an heptagon”).
Han as modified and Stack Overflow A are considered analogous in the art with respect to the use of bounding boxes to define an area of interest. One would be motivated to utilize a bounding box to define an area of interest for machine learning purposes, as is evident in Han as modified, as well as being well known in the art. While Stack Overflow A doesn’t explicitly teach that a midpoint of an edge of a bounding box, opposite the first and second corner, corresponds to a PIG location, a person having ordinary skill in the art before the effective filing date of the claimed invention would understand that because the vertex of the V-shape is indicative of PIG position (Hill, Column 13, lines 13-20: “It can be seen therefore that a pressure pulse is clearly detectable in the pipeline using a distributed acoustic sensor. As movement of a pig in the pipeline will generate a repetitive series of pressure pulses such a repetitive series can be used to locate the pig within the pipeline. Preferably however the characteristic V shape, caused by propagation of the pressure pulse in the both directions within the pipeline, may be used as an acoustic signature of the pig”), the edge of the bounding box corresponds to the location of the PIG where the bounding box is aligned with defined key points, as is demonstrated in Stack Overflow A.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the machine learning model using bounding boxes in waterfall images to determine PIG position of Han as modified with the alignment of a bounding box such that a first and second corner of the bounding box intersect with the V-shaped plot and a PIG position corresponding with the edge opposite to the first and second corner of the bounding box of Stack Overflow A; Doing so would yield the predictable result of clearly establishing the V-shaped plot in the waterfall image, improving the learning process of the machine learning model being trained on the waterfall images.
Regarding Claim 5, the method of Claim 2 is rejected over Han as modified.
Han as modified teaches obtaining waterfall images for machine learning (Han, Summary of the Invention: “representing the vibration intensity of each region with different colors, and then drawing the vibration intensity of each region on the whole optical cable according to time and space into a waterfall diagram; The vibration intensity of different color depths on the waterfall image is taken as the image feature, and deep learning is performed by using the 3D multi-convolutional neural network to determine the traveling position of the cleaning tube detection device”).
Han as modified does not explicitly teach that each bounding box has a fixed width.
However, Stack Overflow A teaches fitting a bounding box to points defining a region of interest (Post by yosmo78: “I want to clip an axis aligned bounding box against a triangle in a way that creates a new tight fitting axis aligned bounding box around or in the triangle”; Refer to images in the post, which demonstrate aligning a bounding box to defined points; Answer by user1196549: “To get a correct result, you need to compute the intersection of the two shapes, for instance using the Sutherland-Hodgman algorithm, that you can specialize for the case of a triangle and a rectangle. If I am right, in the worst case you get an heptagon”).
Han as modified and Stack Overflow A are considered analogous in the art with respect to the use of bounding boxes to define an area of interest. One would be motivated to utilize a bounding box to define an area of interest for machine learning purposes, as is evident in Han as modified, as well as being well known in the art. While Stack Overflow A doesn’t explicitly teach that the bounding box is fixed, A person having ordinary skill in the art before the effective filing date of the claimed invention would understand that given defined points in which a bounding box is aligned to, said bounding box would be fixed, as the defining points, unless specified, are assumed to be unchanging. This is further supported by Han, which obtains waterfall images (which are composed of concrete and unchanging data at a given point of time), and hence a bounding box based on the points in a waterfall image would be fixed, as the points in the waterfall image are unchanging.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the use of bounding boxes in training a machine learning model determining PIG position from static waterfall images of Han as modified with the fitting of a bounding box to defined points of interest of Stack Overflow A; Doing so would yield the predictable result of fixed bounding boxes (including fixed width) based on the points in the static waterfall image.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Han (CN117392401 A), in view of Ling (Data Modeling Techniques for Pipeline Integrity Assessment: A State-of-the-Art Survey, 2023), Ozharar (US 20230130788 A1), Hill (US 8973444 B2) and Munoz (Finding Well-Coupled Optical Fiber Locations for Railway Monitoring Using Distributed Acoustic Sensing, 2023, 2023), and in further view of Stack Overflow B (Confidence value of each Bounding Box, 2016).
Regarding Claim 6, the method of Claim 2 is rejected over Han as modified.
Han as modified does not explicitly teach that training the machine learning model comprises determining a respective confidence level of detection for each bounding box.
However, Stack Overflow B teaches that training a machine learning model comprises determining a respective confidence level of detection for each bounding box (Post by Kapes: “I am currently working on Image classification given an image with neural network. I have successfully created the bounding boxes over the image and for each bounding box I have applied classification algorithm(W^X+B where W, B are weights and biases already learned from training data) to get a value for each of 20 classes”… “How to get this confidence value for each bounding box ? Is it just the probability of class 11 with respect to all other classes? In this data how do I get it”; Answer by Alex I: “To convert the output of a neural network to probabilities, softmax is usually used”).
Han as modified and Stack Overflow B are considered analogous in the art with respect to the use of bounding boxes for determining areas of interest for machine learning purposes. It is well known in the art to classify the image matter within the bounding box according to confidence values, as is evident in Stack Overflow B; Doing so enables classification of matter within bounding boxes at a standard set by the confidence value, which is well known in the art.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the machine learning model identifying PIG position from waterfall images and bounding boxes of Han as modified with the determining of confidence levels of detection for each bounding box of Stack Overflow B; Doing so would yield the predictable result of improving the accuracy of the machine learning model.
Claims 10 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Han (CN117392401 A), in view of Ling (Data Modeling Techniques for Pipeline Integrity Assessment: A State-of-the-Art Survey, 2023) and Stajanca (Detection of Leak-Induced Pipeline Vibrations Using Fiber—Optic Distributed Acoustic Sensing, 2018), and in further view of Ozharar (US 20230130788 A1), Hill (US 8973444 B2) and Munoz (Finding Well-Coupled Optical Fiber Locations for Railway Monitoring Using Distributed Acoustic Sensing, 2023, 2023).
Regarding Claim 10, the system of Claim 9 is rejected over Han as modified.
Han as modified teaches data comprising a waterfall image for training the machine learning model (Han, Summary of the Invention: “representing the vibration intensity of each region with different colors, and then drawing the vibration intensity of each region on the whole optical cable according to time and space into a waterfall diagram; The vibration intensity of different color depths on the waterfall image is taken as the image feature, and deep learning is performed by using the 3D multi-convolutional neural network to determine the traveling position of the cleaning tube detection device”), and
Providing the training images to the machine learning model during a training cycle (Han, Summary of the Invention: “representing the vibration intensity of each region with different colors, and then drawing the vibration intensity of each region on the whole optical cable according to time and space into a waterfall diagram; The vibration intensity of different color depths on the waterfall image is taken as the image feature, and deep learning is performed by using the 3D multi-convolutional neural network to determine the traveling position of the cleaning tube detection device”. Notes: because deep learning is performed using the waterfall image (training image), using a neural network, a training cycle is necessarily performed because learning is performed).
Han as modified does not teach splitting the waterfall images into a plurality of sub-images, and does not teach generating a plurality of respective training images.
However, Ozharar teaches splitting the waterfall images into a plurality of sub-images, and generating a plurality of respective training images (Paragraph [0096]: “In the inference phase, the input waterfall images are converted into local patches with overlaps by sliding window. The patches are then classified by the trained TRN model to determine if an open or close event exists within the patch. The AI engine takes continuous streams of input data and runs inference in real-time on GPU, providing timestamps, event types, and confidence scores as the output”).
Han as modified and Ozharar are considered analogous in the art with respect to the use of waterfall images obtained from fiber optics for machine learning. One would be motivated to split a waterfall image into multiple sub images for training purposes by identifying whether certain fiber optic events occur in a particular patch, as is evident in Ozharar.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the machine learning model using waterfall images of Han as modified with the splitting of waterfall images into a plurality of sub-images for generating training images of Ozharar; Doing so would yield the predictable result of producing more specific training images isolating events of interest, improving the performance of the machine learning model.
Han as modified does not explicitly teach that each training image comprises a V-shaped plot.
However, Hill teaches establishing a window such that each training image comprises a V-shaped plot (Column 12, lines 60-67: “Two main features can be seen from the waterfall plot. The first is an area of constant acoustic disturbance towards the left of the plot at 402, corresponding to a length of approximately 4000 m of the sensing fibre. This is attributable to an industrial unit located over that section of fibre, producing a steady vibrational noise. Secondly distinct chevron (i.e. V shaped) patterns can be seen, most clearly in region 404, away from the constant noise of the industrial unit”; Column 13, lines 1-12: “The vertex of each chevron is located at point 406 along the fibre, corresponding to the origin of the pressure pulse. The `V` shape of the plot corresponds to the pressure pulse moving along the pipe in both directions away from the source of the pulse, and the slope of the `V` shape corresponds to the speed of sound in the pressurised fluid contained within the pipe which in this case is approximately 400 ms.sup.-1. It can be seen that a series of pressure pulses are introduced into the fluid, in this instance pressurized gas, and multiple traces are formed. On the top histogram plot, the individual pulses appear in their respective positions at that instant, spaced along the fibre”; Column 13, lines 37-39: “To simplify the detection a window of data is chosen such that only one pressure pulse is expected during the window”)
Han as modified and Hill are considered analogous in the art with respect to the derivation of PIG position via waterfall images. A common motivation in the art is to isolate the data into specific windows depicting the position of a PIG at an instance of time, as is evident in Hill.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the machine learning model using waterfall images for determining PIG positions of Han as modified with the isolation of waterfall images and their pulses on waterfall images into windows where only a single pulse (V-shaped chevron) is present; Doing so would result in the predictable result of isolating instances in time indicating PIG position in waterfall images, enabling a machine learning model to learn off of specific instances of data, improving the learning process.
Han as modified does not teach that training the machine learning model comprises using bounding boxes centered on the V-shaped plot for training.
However, Munoz teaches data comprising a waterfall image (Figure 2), and using bounding boxes centered on the V-shaped plot for identifying a position of interest (Figure 2).
Han as modified and Munoz are considered analogous in the art with respect to the use of data obtained via fiber optics represented as waterfall images. While Munoz does not explicitly teach using the bounding boxes centered on the V-shaped plot for training a machine learning model, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that training a machine learning model off images comprises using bounding boxes. Doing so is well established in the art as identifying areas of interest for identification by the model. Considering that Munoz identifies the V-shaped plot within the bounding box as being an area of interest with regards to the position of traveling entities, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention that training a model to identify a position of a tracked entity (such as in Han as modified, which itself utilizes waterfall images) would utilize machine learning techniques utilizing waterfall images (such as that of Munoz, which identifies key positions of interest regarding position of entities in the waterfall image).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the training of a machine learning model using a waterfall image of Han as modified with the use of bounding boxes centered on the V-shaped plot for training the machine learning model of Munoz; Doing so would yield the predictable result of training a machine learning model through the established use of images annotated with bounding boxes for identifying regions of interest.
Claim 17, being similar in scope to Claim 10, is rejected under the same rationale.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Han (CN117392401 A), in view of Ling (Data Modeling Techniques for Pipeline Integrity Assessment: A State-of-the-Art Survey, 2023) and Stajanca (Detection of Leak-Induced Pipeline Vibrations Using Fiber—Optic Distributed Acoustic Sensing, 2018), and in further view of Hill (US 8973444 B2).
Regarding Claim 20, the non-transitory, computer readable medium of Claim 16 is rejected over Han as modified.
Han as modified does not explicitly teach contextualizing the fiber optic events is based on data from an inspection report, a simulation, a supervisory control and data acquisition (SCADA) system, a historical report, or any combination thereof.
However, Hill teaches contextualizing the fiber optic events based on a data from a historical report (Column 12, lines 60-67: “Two main features can be seen from the waterfall plot. The first is an area of constant acoustic disturbance towards the left of the plot at 402, corresponding to a length of approximately 4000 m of the sensing fibre. This is attributable to an industrial unit located over that section of fibre, producing a steady vibrational noise. Secondly distinct chevron (i.e. V shaped) patterns can be seen, most clearly in region 404, away from the constant noise of the industrial unit”. Notes: a waterfall image generated based on the data obtained from the fiber optic cables is considered to be historical data since each waterfall image depicts data overtime. The broadest reasonable interpretation of a report is simply the presentation of data or information in any form).
Han as modified and Hill are considered analogous in the art with respect to the derivation of PIG position via waterfall images. A common motivation in the art is to utilize waterfall images for determining fiber optic events, as is evident in Hill.
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to combine the waterfall images used for determining PIG position of Han as modified with the contextualizing information present in waterfall images of Hill; Doing so would yield the predictable result of enabling the determination of PIG position from waterfall images.
Allowable Subject Matter
Claims 7-8, 11-15 and 18-19 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Claims 7-8 are deemed to be allowable subject matter dependent on rejected base Claims 1-4.
Regarding Claim 7, while running a linear fitting algorithm to generate a trajectory of a PIG in a pipeline is well established, there is no prior art that teaches a polygon fitting algorithm for generating a trajectory of a PIG. Furthermore, there is not a strong case for obviousness regarding a combination of prior art for teaching polygon fitting for generating a trajectory of a PIG.
Huang (Pipeline Inspection Gauge Positioning System Based on Optical Fiber Distributed Acoustic Sensing, 2021) teaches performing a linear fit on a waterfall image to obtain trajectory information for a PIG.
However, Huang does not teach or suggest utilizing a polygon fitting algorithm for doing so.
Considering that no prior art teaches or suggests utilizing a polygon fitting algorithm for generating a trajectory of a PIG, and no combination of prior art would be obvious to teach such a limitation, Claim 7 is considered allowable subject matter.
Claim 8, being dependent on Claim 7, is objected to as being allowable subject matter.
Claims 7-8, upon being rewritten into independent Claim 1 along with base claims 2-4, would be considered allowable.
Claim 11, being similar in scope to Claim 7, is similarly objected to as being allowable subject matter. Additionally, Claims 12-15, being dependent on Claim 11, are also objected to as being allowable subject matter.
Claims 11-15, upon being rewritten into independent Claim 9 along with base claim 10, which Claims 11-15 depend upon or ultimately depend upon, would be considered allowable.
Claim 18, being similar in scope to Claim 7, is similarly objected to as being allowable subject matter. Additionally, Claim 19, being dependent on Claim 18, is also objected to as being allowable subject matter.
Claims 18-19, upon being rewritten into independent Claim 16 along with base claim 17, which Claims 18-19 depend upon or ultimately depend upon, would be considered allowable.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RAYMOND CHUN LAM LI whose telephone number is (571)272-5124. The examiner can normally be reached M-F 8:30-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kent Chang can be reached at 571-272-7667. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RAYMOND CHUN LAM LI/Examiner, Art Unit 2614
/KENT W CHANG/Supervisory Patent Examiner, Art Unit 2614
Prosecution Insights