DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. This office action is responsive to Applicant’s Arguments/Remarks Made in an Amendment received on 12/22/2025.
Drawings
3. It is noted that the drawings have been amended. Objection(s) with respect to the drawing(s) have been fully considered and are withdrawn.
Claim Objections
4. In view of Applicant’s argument [Remarks] and amendments filed 12/22/2025, claim objection(s) with respect to claims 17, 18 and 19 have been fully considered and the objection of claims 17, 18 and 19 are withdrawn.
Status of Claims
5. Claims 1-20 are pending in this application.
Claims 1, 12, and 17-20 are currently amended.
Response to Arguments
6. Regarding Applicant’s Argument (pages 9-10):
Applicant contends that McCrackin merely discloses “current inspection images and historical inspection images of the same region,” and that such images are not “data characterizing an image recognition library that includes example mechanical equipment features.” Applicant further asserts that McCrackin “neither discloses nor implies” that data characterizing images of the inspection region or historical images of the inspection region characterize an image recognition library.
These arguments are not persuasive. McCrackin describes an analytical model that includes an image recognition algorithm configured to identify a “site feature” (e.g., a defect type) in images of an inspection region of an industrial machine. (McCrackin: ¶¶[0003-0005; 0025; 0029]). Paragraph [0005] explains that “determining one or more characteristic includes … determining an identity of the site feature by an image recognition algorithm in the analytical model,” and further that “the image recognition algorithm receives data characterizing the one or more images of the inspection region and historical images of [the] inspection region as inputs.” Paragraph [0029] describes that the “image recognition algorithm can be trained on training data including images of multiple site features in multiple inspection regions (e.g., different types of defects in multiple industrial machines),” and that “a previously trained image recognition algorithm can be stored in a memory device accessible to the controller.” McCrackin further specifies that the training data include images of many defect types (crack, tear, rub, dent, coating loss, missing material, erosion, excess material, fissure, etc.) across various industrial machines (turbines, automotive engines, heat exchangers, industrial piping, etc.).
The trained image recognition algorithm stored in memory, together with its associated learned parameters/features derived from the training images of multiple defect types and machine parts, constitute an image recognition library including example mechanical equipment features, within the broad scope of the claim language “image recognition library that includes example mechanical equipment features.” Nothing in claim 1 requires that the “library” be a separate, human labeled database; the claim merely recites a library used for image recognition that includes example features, which is reasonably read on McCrackin’s stored trained model and its feature data.
Additionally, McCrackin teaches that the controller receives “inspection data” from the inspection device, where “the received data can include … acquired images, acquired videos, etc.,” as well as characteristics of the inspection site, and that such inspection data can be transmitted to a computing device for analysis or stored in a data file. (McCrackin: ¶[0028]).
McCrackin further discloses generating a data file and/or notification that includes “data characterizing the one or more images of the inspection region, the one or more characteristics of the inspection region, the identity of the site feature, image quality characteristics, site feature characteristics, and the new set of operating parameters,” where this data file can be saved in a database (e.g., in the controller, computing device, or cloud database) and linked to the site feature for later use during inspection. (McCrackin: ¶¶[0012-0013; 0041-0042]). These passages describe storing, for each site feature, inspection images and analysis results that together provide example instances of mechanical equipment features (e.g., particular defects on particular machines) for use by the system in subsequent inspections.
Accordingly, McCrackin teaches receiving and using data that characterizes an image recognition library comprising example mechanical‑equipment features, namely, the stored trained image recognition algorithm and associated feature data, as well as the stored inspection data files linked to identified site features. This corresponds to the claimed “second set of data characterizing an image recognition library that includes example mechanical equipment features.” Therefore, the amendment to claim 1 does not patentably distinguish over McCrackin, and the rejection of claim 1 under 35 U.S.C. 102(a)(1) is properly maintained and made final.
Claim Rejections - 35 USC § 102
7. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
8. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
9. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
10. Claims 1, 3-5, 9-10, 12, 14-16 and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by McCrackin et al. (US 2022/0132040).
Regarding Claim 1:
McCrackin discloses a method (McCrackin: ‘method for inspecting an inspection site (e.g., an industrial machine)’ [0025]) comprising:
receiving a first set of data characterizing an image or video captured by a non- destructive testing device from within mechanical equipment during an inspection, and
McCrackin describes receiving data characterizing one or more images of an inspection region of an industrial machine acquired by an inspection system that includes a nondestructive testing device such as a borescope. (McCrackin: Abstract; ¶¶[0003; 0005; 0024-0026]). The inspection system acquires images and/or video of the inspection region, and controller 204 receives inspection data including acquired images and acquired videos from inspection device 202. (McCrackin: ¶[0028]). Thus, McCrackin teaches receiving a first set of data characterizing an image or video captured by a non‑destructive testing device from within mechanical equipment during an inspection.
a second set of data characterizing an image recognition library that includes example mechanical equipment features;
McCrackin further teaches that an analytical model includes an image recognition algorithm configured to determine an identity of a site feature (e.g., defect type) in the inspection region. (McCrackin: ¶¶[0005; 0029]). Paragraph [0029] explains that the “image recognition algorithm can be trained on training data including images of multiple site features in multiple inspection regions (e.g., different types of defects in multiple industrial machines such as turbines, automotive engines, heat exchangers, industrial piping, etc.),” and that “a previously trained image recognition algorithm can be stored in a memory device accessible to the controller.” The trained image recognition algorithm stored in memory, with its learned parameters and feature information derived from the training images of multiple defect types on different industrial machines, constitutes an image recognition library that includes example mechanical‑equipment features.
Additionally, McCrackin teaches generating and storing a data file that includes data characterizing images of the inspection region, the identity of the site feature, image quality characteristics, site feature characteristics, and operating parameters, where this data file is saved in a database and linked to the respective site feature for later use during inspections. (McCrackin: ¶¶[0012-0013; 0041-0042]). These stored data files provide example instances of mechanical‑equipment features (e.g., particular defects on particular machine parts) for use by the system in subsequent inspections. Accordingly, the inspection system receives and uses a second set of data that characterizes an image recognition library including example mechanical‑equipment features, as claimed.
determining a presence of a feature based on the first set of received data and the second set of received data; and
McCrackin states that the analytical model determines one or more characteristics of the inspection region from the received data, including determining an identity of the site feature by an image‑recognition algorithm that receives data characterizing one or more images of the inspection region and historical images of the inspection region as inputs. (McCrackin: ¶¶[0005; 0029-0031]). The algorithm uses the trained model (image recognition library) together with the received inspection images to identify the defect type and other site‑feature characteristics, thereby determining the presence and type of a site feature based on both the current inspection data and the stored library data
providing an indication of a presence of the feature
McCrackin further discloses generating a notification and/or data file that includes, inter alia, data characterizing the image of the inspection region, the identity of the site feature, image quality characteristics, site feature characteristics, and recommendations for further inspection, and presenting this information to a user via a graphical user interface. (McCrackin: ¶¶[0012-0013, 0041-0042]). This presentation/notification constitutes providing an indication of the presence (and type) of the site feature, as claimed.
Regarding Claim 3:
McCrackin further discloses the method of claim 1, wherein providing the indication comprises:
emitting an audible signal to a user (McCrackin: ‘feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback)’ [0058]); or
providing haptic feedback to the user (McCrackin: ‘feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback)’ [0058]; Claim Interpretation: wherein ‘tactile feedback’ is interpreted as a type/subset of the claimed ‘haptic feedback’).
Regarding Claim 4:
McCrackin further discloses the method of claim 1, further comprising:
receiving a user input indicative of a request to see more details (McCrackin: ‘receiving an input from the user in response to the presented notification; and generating the control signal based on the received input. The new inspection of the inspection region by the inspection system is based on the control signal.’ [0013; 0041]);
determining a location of the feature present within the image or video (McCrackin: ‘the notification can include an inspection summary that can include data characterizing the image received at step 102, the characteristics of the inspection region determined at step 104, the new inspection including the new operating parameters, etc.’ [0042]; [0013-0016]);
displaying the image or video upon a display screen (McCrackin: ‘The notification can be provided to an operator/user. This can be done, for example, via a GUI display space in a computing device (e.g., a laptop, a tablet, computing device 206, controller 204, etc.)’ [0042]); and
displaying a graphical object overlaid onto the image or video, the graphical object being overlaid to indicate a portion of the image or video containing the feature (McCrackin: ‘The notification can include modified images of the inspection region that include image analysis information (e.g., markers around site features, inspection data, etc.)’ [0042]).
Regarding Claim 5:
The method of claim 4, wherein the user input comprises an input on a touch screen, a voice command, or a press of an electromechanical switch (McCrackin: ‘feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input.’ [0058]).
Regarding Claim 9:
McCrackin further discloses the method of claim 1, wherein receiving the first set of data and the second set of data comprises receiving the first set of data and the second set of data by a borescope (McCrackin: ‘The inspection system can include, for example, an inspection device (e.g., a borescope)’ [0025]; Fig. 3 ‘boroscope 300’).
Regarding Claim 10:
McCrackin further discloses the method of claim 1, wherein the feature is a defect (McCrackin: ‘the site feature is a site defect in the inspection region of the industrial machine. Identifying the site feature includes determining a defect type associated with the site defect.’ [0005]).
Regarding Claim 12:
McCrackin discloses a system (McCrackin: Fig. 2 ‘inspection system configured to inspect an inspection site (e.g., an industrial machine)’ [0026]) comprising:
at least one data processor (McCrackin: Fig. 2 ‘computing device 206, controller 204’); and
non-transitory memory storing instructions, which, when executed by the at least one data processor causes the at least one data processor (McCrackin: ‘computer program instructions stored on memory executed by a processor’ [0056-0057]) to perform operations comprising:
receiving a first set of data characterizing an image or video captured by a non-destructive testing device from within mechanical equipment during an inspection, and
As discussed for claim 1, McCrackin discloses an analytical model including an image‑recognition algorithm that is trained on images of multiple site features in multiple inspection regions of various industrial machines and stored in a memory device accessible to the controller. (McCrackin: ¶¶[0005; 0029]). The trained algorithm, with its learned parameters and feature information derived from these example defects and machine parts, constitutes an image recognition library including example mechanical‑equipment features.
a second set of data characterizing an image recognition library that includes example mechanical equipment features;
McCrackin’s As discussed for claim 1, McCrackin discloses an analytical model including an image recognition algorithm that is trained on images of multiple site features in multiple inspection regions of various industrial machines and stored in a memory device accessible to the controller. (McCrackin: ¶¶[0005; 0029]). The trained algorithm, with its learned parameters and feature information derived from these example defects and machine parts, constitutes an image recognition library including example mechanical‑equipment features.
determining a presence of a feature based on the first set of received data and the second set of received data; and
McCrackin states that the analytical model (executed by the controller) determines one or more characteristics of the inspection region, including determining an identity of the site feature via the image‑recognition algorithm, which receives data characterizing images of the inspection region and historical images of the inspection region as inputs. (McCrackin: ¶¶[0005; 0025; 0029-0031]). The identity and characteristics of the site feature are determined using the trained model (image recognition library) together with the received inspection images, thereby corresponding to determining a presence of a feature based on both the first and second sets of data.
providing an indication of a presence of the feature.
McCrackin teaches that controller 204 and/or computing device 206 generates a notification and/or data file that includes data characterizing the images of the inspection region, the identity of the site feature, image quality characteristics, site feature characteristics, and recommendations for further inspection, and that this notification/data file is presented to the user via a GUI. (McCrackin: ¶¶[0012–0013; 0041-0042]). Presenting this notification, including the identity and characteristics of the site feature, constitutes providing an indication of the presence of the feature
Regarding Claim 14:
McCrackin further discloses the system of claim 12, wherein providing the indication comprises:
emitting an audible signal to a user (McCrackin: ‘feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback)’ [0058]); or
providing haptic feedback to the user (McCrackin: ‘feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback)’ [0058]; Claim Interpretation: wherein ‘tactile feedback’ is interpreted as a type/subset of the claimed ‘haptic feedback’).
Regarding Claim 15:
McCrackin further discloses the system of claim 12, wherein the operations further comprise:
receiving a user input indicative of a request to see more details (McCrackin: ‘receiving an input from the user in response to the presented notification; and generating the control signal based on the received input. The new inspection of the inspection region by the inspection system is based on the control signal.’ [0013; 0041]);
determining a location of the feature present within the image or video (McCrackin: ‘the notification can include an inspection summary that can include data characterizing the image received at step 102, the characteristics of the inspection region determined at step 104, the new inspection including the new operating parameters, etc.’ [0042]; [0013-0016]);
displaying the image or video upon a display screen (McCrackin: ‘The notification can be provided to an operator/user. This can be done, for example, via a GUI display space in a computing device (e.g., a laptop, a tablet, computing device 206, controller 204, etc.)’ [0042]); and
displaying a graphical object overlaid onto the image or video, the graphical object being overlaid to indicate a portion of the image or video containing the feature (McCrackin: ‘The notification can include modified images of the inspection region that include image analysis information (e.g., markers around site features, inspection data, etc.)’ [0042]).
Regarding Claim 16:
McCrackin further discloses the system of claim 15, wherein the user input comprises an input on a touch screen, a voice command, or a press of an electromechanical switch (McCrackin: ‘feedback provided to the user can be any form of sensory feedback, (e.g., visual feedback, auditory feedback, or tactile feedback), and input from the user can be received in any form, including acoustic, speech, or tactile input.’ [0058]).
Regarding Claim 20: (drawn to a computer-readable storage medium)
The proposed rejection of method claim 1, and system claim 12, over McCrackin is similarly cited to reject the computer readable medium of claim 20 because these steps occur in the operation of the method and system as discussed above. Thus, the arguments similar to that presented above for claims 1 and 12 are equally applicable to claim 20.
It is noted that McCrackin expressly states that “non‑transitory computer program products … store instructions, which when executed by one or more data processors … cause at least one data processor to perform operations herein,” and that “methods can be implemented by one or more data processors” in computing systems connected by a network. See ¶[0018].
Claim Rejections - 35 USC § 103
11. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
12. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
13. The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
14. The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or
15. Claims 2 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over McCrackin et al. (US 2022/0132040) in view of Badawy et al. (US 10,373,470).
Regarding Claim 2:
McCrackin discloses the method of claim 1, but does not expressly disclose wherein providing the indication comprises displaying a flashing border on a graphical user interface displayed, with the image or video, on display screen.
Badawy discloses wherein providing the indication comprises displaying a flashing border on a graphical user interface displayed, with the image or video, on display screen (Badawy: ‘wherein providing the indication comprises displaying a flashing border on a graphical user interface displayed, with the image or video, on display screen.’ Col. 7, line 62 – Col. 8, line 2).
McCrackin in view of Badawy are combinable because they are from the same field of endeavor of image processing; e.g., both disclose methods of detecting and classifying objects and alerting a user/operator if there is an anomaly.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein providing the indication comprises displaying a flashing border on a graphical user interface displayed, with the image or video, on display screen.
The suggestion/motivation for doing so is to trigger an alert message if for example, a leak is detected as disclosed by Badawy in the Background of Invention. Therefore, it would have been obvious to combine McCrackin with Badawy to obtain the invention as specified in claim 2.
Regarding Claim 13:
McCrackin discloses the system of claim 12, but does not expressly disclose wherein providing the indication comprises displaying a flashing border on a graphical user interface displayed, with the image or video, on display screen.
Badawy discloses wherein providing the indication comprises displaying a flashing border on a graphical user interface displayed, with the image or video, on display screen (Badawy: ‘wherein providing the indication comprises displaying a flashing border on a graphical user interface displayed, with the image or video, on display screen.’ Col. 7, line 62 – Col. 8, line 2).
McCrackin in view of Badawy are combinable because they are from the same field of endeavor of image processing; e.g., both disclose methods of detecting and classifying objects and alerting a user/operator if there is an anomaly.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein providing the indication comprises displaying a flashing border on a graphical user interface displayed, with the image or video, on display screen.
The suggestion/motivation for doing so is to trigger an alert message if for example, a leak is detected as disclosed by Badawy in the Background of Invention. Therefore, it would have been obvious to combine McCrackin with Badawy to obtain the invention as specified in claim 13.
16. Claims 6-8, and 17-19 are rejected under 35 U.S.C. 103 as being unpatentable over McCrackin et al. (US 2022/0132040) in view of Messinger et al. (US 9,710,573).
Regarding Claim 6:
McCrackin further discloses the method of claim 4, wherein the user input is a first user input, wherein the graphical object is a first graphical object, the method further comprising:
receiving a second user input indicative of a request to see more details (McCrackin; ‘This process can be repeated resulting in inspection of the device in multiple positions/orientation.’ [0027]);
determining a type of feature present within the image or video (McCrackin: ‘the notification can include an inspection summary that can include data characterizing the image received at step 102, the characteristics of the inspection region determined at step 104, the new inspection including the new operating parameters, etc.’ [0042]; [0013-0016]);
McCrackin does not expressly disclose displaying a second graphical object overlaid onto the image or video, the second graphical object being overlaid onto the image or video adjacent to the first graphical object, the second graphical object configured to convey the determined type of feature to a user.
Messinger discloses displaying a second graphical object overlaid onto the image or video, the second graphical object being overlaid onto the image or video adjacent to the first graphical object, the second graphical object configured to convey the determined type of feature to a user (Messinger: Fig. 6 ‘The NDT inspection device 12 may, in addition to capturing images, overlay certain data onto the image, resulting in a more informative view... A variety of overlays may be provided, as described in more detail below, including measurement overlays, menu overlays, annotation overlays, and object identification overlays. The image and video data, such as the video 84, may then be displayed, with the overlays generally displayed on top of the image and video data’ Col. 11, lines 46-61; ‘Overlay data 168 may also include annotation data. For example, text and graphics (e.g. arrow pointers, crosses, geometric shapes) may be overlaid on top of an image to annotate certain features, such as “surface crack.” Additionally, audio may be captured by the NDT inspection device 12, and provided as an audio overlay. For example, a voice annotation, sounds of the equipment undergoing inspection, and so on, may be overlaid on an image or video as audio.’ Col. 13, lines 20-27; ‘sensor data 174 may be synchronized with the overlay data 168, for example, overlay tip maps may be displayed alongside with temperature information, pressure information, flow information, clearance, and so on. Likewise, the sensor data 174 may be displayed alongside the image or video data 170.’ Col. 13, lines 43-48’).
McCrackin in view of Messinger are combinable because they are from the same field of endeavor of image processing; e.g. both disclose methods of detecting and classifying objects and alerting a user/operator if there is an anomaly such as defects or cracks.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to displaying a second graphical object overlaid onto the image or video, the second graphical object being overlaid onto the image or video adjacent to the first graphical object, the second graphical object configured to convey the determined type of feature to a user. The suggestion/motivation for doing so is to improve data inspection and analysis as disclosed by Messinger at Col. 12, lines 10-11. Therefore, it would have been obvious to combine McCracking with Messinger to obtain the invention as specified in claim 6.
Regarding Claim 7:
The proposed combination of McCrackin in view of Messinger further discloses the method of claim 6, further comprising:
receiving third a user input acknowledging the determined location of the feature and the determined type of feature details (McCrackin; ‘The notification can be presented to a user who can review the notification and provide an input. The input from the user can include an approval for the execution of the new inspection (e.g., based on the new set of operating parameters). Alternately, the user can make changes to the new inspection (e.g., by changing one or more of the new set of operating parameters).’ [0041]; ‘This process can be repeated resulting in inspection of the device in multiple positions/orientation.’ [0027]);
saving the determined location of the feature and the determined type of the feature (McCrackin: ‘The notification can provide an option to the user to save one or more of the inspection data associated with the site feature, operating parameters/constraints of the inspection device associated with the inspection of the site feature, and additional notes. Upon receiving instructions from the user, a data file including the aforementioned data can be generated and saved (e.g., in a database in controller 204, computing device 206, cloud database, etc.).’ [0042]); and
displaying the image or video without the first graphical object or the second graphical object (Messinger: Messinger discloses The NDT inspection device 12 may, in addition to capturing images, overlay certain data onto the image, resulting in a more informative view. This implies that any overlays that can be provided do not have to be used (Messinger at Col. 11, lines 46-48); Further, Fig. 6 ‘The NDT inspection device 12 may, in addition to capturing images, overlay certain data onto the image, resulting in a more informative view... A variety of overlays may be provided, as described in more detail below, including measurement overlays, menu overlays, annotation overlays, and object identification overlays. The image and video data, such as the video 84, may then be displayed, with the overlays generally displayed on top of the image and video data’ Col. 11, lines 46-61; ‘Overlay data 168 may also include annotation data. For example, text and graphics (e.g. arrow pointers, crosses, geometric shapes) may be overlaid on top of an image to annotate certain features, such as “surface crack.” Additionally, audio may be captured by the NDT inspection device 12, and provided as an audio overlay. For example, a voice annotation, sounds of the equipment undergoing inspection, and so on, may be overlaid on an image or video as audio.’ Col. 13, lines 20-27; ‘sensor data 174 may be synchronized with the overlay data 168, for example, overlay tip maps may be displayed alongside with temperature information, pressure information, flow information, clearance, and so on. Likewise, the sensor data 174 may be displayed alongside the image or video data 170.’ Col. 13, lines 43-48’).
McCrackin in view of Messinger are combinable because they are from the same field of endeavor of image processing; e.g. both disclose methods of detecting and classifying objects and alerting a user/operator if there is an anomaly such as defects or cracks.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to receiving third a user input acknowledging the determined location of the feature and the determined type of feature details; saving the determined location of the feature and the determined type of the feature; and displaying the image or video without the first graphical object or the second graphical object. The suggestion/motivation for doing so is to improve the filtering of inspection data are desirable as disclosed by Messinger in the Background of Invention. Therefore, it would have been obvious to combine McCrackin with Messinger to obtain the invention as specified in claim 7.
Regarding Claim 8:
The proposed combination of McCrackin in view of Messinger further discloses the method of claim 6, further comprising:
receiving a third user input indicative that the determined location or the determined type are to be changed (McCrackin; ‘The notification can be presented to a user who can review the notification and provide an input. The input from the user can include an approval for the execution of the new inspection (e.g., based on the new set of operating parameters). Alternately, the user can make changes to the new inspection (e.g., by changing one or more of the new set of operating parameters).’ [0041]; ‘This process can be repeated resulting in inspection of the device in multiple positions/orientation.’ [0027]); updating the determined location or the determined type; and providing the updated determined location or the determined type (McCrackin: ‘The data file can include one or more of data characterizing the one or more images of the inspection region, the one or more characteristics of the inspection region, the identity of the site feature, image quality characteristics, site feature characteristics, and the new set of operating parameters. The generated data file can be linked to the site feature (e.g., as an icon with an indication of the site feature)’ [0042]).
McCrackin in view of Messinger are combinable because they are from the same field of endeavor of image processing; e.g. both disclose methods of detecting and classifying objects and alerting a user/operator if there is an anomaly such as defects or cracks.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to receiving a third user input indicative that the determined location or the determined type are to be changed; updating the determined location or the determined type; and providing the updated determined location or the determined type. The suggestion/motivation for doing so is to improve the filtering of inspection data are desirable as disclosed by Messinger in the Background of Invention. Therefore, it would have been obvious to combine McCrackin with Messinger to obtain the invention as specified in claim 8.
Regarding Claim 17:
McCrackin further discloses the system of claim 15, wherein the operations further comprise:
receiving a second user input indicative of a request to see more details (McCrackin; ‘This process can be repeated resulting in inspection of the device in multiple positions/orientation.’ [0027]);
determining a type of feature present within the image or video (McCrackin: ‘the notification can include an inspection summary that can include data characterizing the image received at step 102, the characteristics of the inspection region determined at step 104, the new inspection including the new operating parameters, etc.’ [0042]; [0013-0016]);
McCrackin does not expressly disclose displaying a second graphical object overlaid onto the image or video, the second graphical object being overlaid onto the image or video adjacent to the first graphical object, the second graphical object configured to convey the determined type of feature to a user.
Messinger discloses displaying a second graphical object overlaid onto the image or video, the second graphical object being overlaid onto the image or video adjacent to the first graphical object, the second graphical object configured to convey the determined type of feature to a user (Messinger: Fig. 6 ‘The NDT inspection device 12 may, in addition to capturing images, overlay certain data onto the image, resulting in a more informative view... A variety of overlays may be provided, as described in more detail below, including measurement overlays, menu overlays, annotation overlays, and object identification overlays. The image and video data, such as the video 84, may then be displayed, with the overlays generally displayed on top of the image and video data’ Col. 11, lines 46-61; ‘Overlay data 168 may also include annotation data. For example, text and graphics (e.g. arrow pointers, crosses, geometric shapes) may be overlaid on top of an image to annotate certain features, such as “surface crack.” Additionally, audio may be captured by the NDT inspection device 12, and provided as an audio overlay. For example, a voice annotation, sounds of the equipment undergoing inspection, and so on, may be overlaid on an image or video as audio.’ Col. 13, lines 20-27; ‘sensor data 174 may be synchronized with the overlay data 168, for example, overlay tip maps may be displayed alongside with temperature information, pressure information, flow information, clearance, and so on. Likewise, the sensor data 174 may be displayed alongside the image or video data 170.’ Col. 13, lines 43-48’).
McCrackin in view of Messinger are combinable because they are from the same field of endeavor of image processing; e.g. both disclose methods of detecting and classifying objects and alerting a user/operator if there is an anomaly such as defects or cracks.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to displaying a second graphical object overlaid onto the image or video, the second graphical object being overlaid onto the image or video adjacent to the first graphical object, the second graphical object configured to convey the determined type of feature to a user. The suggestion/motivation for doing so is to improve data inspection and analysis as disclosed by Messinger at Col. 12, lines 10-11. Therefore, it would have been obvious to combine McCracking with Messinger to obtain the invention as specified in claim 17.
Regarding Claim 18:
The proposed combination of McCrackin in view of Messinger further discloses the system of claim 17, wherein the operations further comprise:
receiving a third user input acknowledging a determined location of the feature and a determined type of feature (McCrackin; ‘The notification can be presented to a user who can review the notification and provide an input. The input from the user can include an approval for the execution of the new inspection (e.g., based on the new set of operating parameters). Alternately, the user can make changes to the new inspection (e.g., by changing one or more of the new set of operating parameters).’ [0041]; ‘This process can be repeated resulting in inspection of the device in multiple positions/orientation.’ [0027]);
saving the determined location of the feature and the determined type of the feature (McCrackin: ‘The notification can provide an option to the user to save one or more of the inspection data associated with the site feature, operating parameters/constraints of the inspection device associated with the inspection of the site feature, and additional notes. Upon receiving instructions from the user, a data file including the aforementioned data can be generated and saved (e.g., in a database in controller 204, computing device 206, cloud database, etc.).’ [0042]); and
displaying the image or video without the first graphical object or the second graphical object (Messinger: Messinger discloses The NDT inspection device 12 may, in addition to capturing images, overlay certain data onto the image, resulting in a more informative view. This implies that any overlays that can be provided do not have to be used (Messinger at Col. 11, lines 46-48); Further, Fig. 6 ‘The NDT inspection device 12 may, in addition to capturing images, overlay certain data onto the image, resulting in a more informative view... A variety of overlays may be provided, as described in more detail below, including measurement overlays, menu overlays, annotation overlays, and object identification overlays. The image and video data, such as the video 84, may then be displayed, with the overlays generally displayed on top of the image and video data’ Col. 11, lines 46-61; ‘Overlay data 168 may also include annotation data. For example, text and graphics (e.g. arrow pointers, crosses, geometric shapes) may be overlaid on top of an image to annotate certain features, such as “surface crack.” Additionally, audio may be captured by the NDT inspection device 12, and provided as an audio overlay. For example, a voice annotation, sounds of the equipment undergoing inspection, and so on, may be overlaid on an image or video as audio.’ Col. 13, lines 20-27; ‘sensor data 174 may be synchronized with the overlay data 168, for example, overlay tip maps may be displayed alongside with temperature information, pressure information, flow information, clearance, and so on. Likewise, the sensor data 174 may be displayed alongside the image or video data 170.’ Col. 13, lines 43-48’).
McCrackin in view of Messinger are combinable because they are from the same field of endeavor of image processing; e.g. both disclose methods of detecting and classifying objects and alerting a user/operator if there is an anomaly such as defects or cracks.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to receiving third a user input acknowledging the determined location of the feature and the determined type of feature details; saving the determined location of the feature and the determined type of the feature; and displaying the image or video without the first graphical object or the second graphical object. The suggestion/motivation for doing so is to improve the filtering of inspection data are desirable as disclosed by Messinger in the Background of Invention. Therefore, it would have been obvious to combine McCrackin with Messinger to obtain the invention as specified in claim 18.
Regarding Claim 19:
The proposed combination of McCrackin in view of Messinger further discloses the system of claim 17, wherein the operations further comprise:
receiving a third user input indicative that the determined location or the determined type are to be changed (McCrackin; ‘The notification can be presented to a user who can review the notification and provide an input. The input from the user can include an approval for the execution of the new inspection (e.g., based on the new set of operating parameters). Alternately, the user can make changes to the new inspection (e.g., by changing one or more of the new set of operating parameters).’ [0041]; ‘This process can be repeated resulting in inspection of the device in multiple positions/orientation.’ [0027]); updating the determined location or the determined type; and providing the updated determined location or the determined type (McCrackin: ‘The data file can include one or more of data characterizing the one or more images of the inspection region, the one or more characteristics of the inspection region, the identity of the site feature, image quality characteristics, site feature characteristics, and the new set of operating parameters. The generated data file can be linked to the site feature (e.g., as an icon with an indication of the site feature)’ [0042]).
McCrackin in view of Messinger are combinable because they are from the same field of endeavor of image processing; e.g. both disclose methods of detecting and classifying objects and alerting a user/operator if there is an anomaly such as defects or cracks.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to receiving a third user input indicative that the determined location or the determined type are to be changed; updating the determined location or the determined type; and providing the updated determined location or the determined type. The suggestion/motivation for doing so is to improve the filtering of inspection data are desirable as disclosed by Messinger in the Background of Invention. Therefore, it would have been obvious to combine McCrackin with Messinger to obtain the invention as specified in claim 19.
17. Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over McCrackin as applied to claim 1 above, and further in view of Papadimitriou et al. (US 2016/0237804).
Regarding Claim 11:
McCrackin discloses the method of claim 1, but does not expressly disclose wherein the mechanical equipment comprises a compressor, a turbine, a conduit, a pump, or a pressure vessel.
Papadimitriou discloses wherein the mechanical equipment comprises a compressor, a turbine, a conduit, a pump, or a pressure vessel (Papadimitriou: ‘The list of typical materials and systems includes, but is not limited to,…coiled tubing (herein after referred to as “CT”)…compressor…pipeline…pump,…turbine…’ [0002]; ‘pressure containment vessel’ [0195]; {Claim Interpretation: wherein ‘coiled tubing’ and ‘pipeline’ read on the claimed ‘conduit’}).
McCrackin in view of Papadimitriou are combinable because they are from the same field of endeavor of image processing; e.g., both disclose methods of Non-destructive Testing/Inspection and determining if there is a defect and issuing an alert.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to disclose wherein the mechanical equipment comprises a compressor, a turbine, a conduit, a pump, or a pressure vessel.
The suggestion/motivation for doing so is to trigger an alert if there are safety, operational, functionality and serviceability issues as disclosed by Papadimitriou in the Background of Invention. Therefore, it would have been obvious to combine McCrackin with Papadimitriou to obtain the invention as specified in claim 11.
Conclusion
18. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
19. Any inquiry concerning this communication or earlier communications from the examiner should be directed to NEIL R MCLEAN whose telephone number is (571)270-1679. The examiner can normally be reached Monday-Thursday, 6AM - 4PM, PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Akwasi M Sarpong can be reached at 571.270.3438. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NEIL R MCLEAN/Primary Examiner, Art Unit 2681
Prosecution Insights