DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendment filed March 3rd 2026 has been entered and made of record. Claims 1, 3-4 and 15-17 are amended. Claims 1-20 are pending.
Applicant’s remarks in view of the newly presented amendments have been considered and are persuasive, however new art is presented to teach the newly added claim limitations.
The rejection is accordingly made FINAL as necessitated by the amendment.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over the combination of USPNs 2014/0207406 to Domke et al. and 2022/0236197 to Wang et al.
With regard to claim 1, Domke discloses an augmented reality inspection system for inspecting features within components, comprising:
a camera system configured to capture images of a component (paragraphs [0025], [0049], borescope camera);
a controller comprising a processor (paragraphs [0009], [0028], and [0036], processor is disclosed to perform the detection logic programming) in communication with a memory device storing a model of the component (paragraphs [0028], [0036], [0045], [0059], Memory is disclosed for storing images and detections of the images), wherein the controller is programmed and configured to receive the images from the camera system, utilize the received images to determine a location of the camera system and the captured images relative to the component, and utilize the determined location to generate graphic images associated with a feature of the component at the determined location (paragraph [0049], a borescope tip map can be displayed and overlaid on the display image to give an approximation of the location of the borescope tip as a means to guide the operator); and
a display configured to generate a real-time display of the component and the feature based in the captured images, wherein the controller is further configured to superimpose the graphic images associated with the feature of the component onto the real-time display of the component (paragraphs [0025]-[0026] and [0049], Display is connected to the borescope camera and capable of overlaying certain data onto the image for more informative view).
Domke discloses performing image recognition (paragraphs [0071], [0075]) in order to identify the location of the probe and to inspect objects. Image recognition implies an analysis and comparison to a known image of the object, however Domke does not explicitly recite the added limitation of determine a location of the camera system and the captured images relative to the component by performing an analysis of features within the captured images and correlating the features to the stored model of the component.
Wang discloses a similar borescope inspection method and device and teaches that object detection is performed and that the object detection may be performed with convolutional neural networks trained for specific parts of the engine being inspected, meaning that the detected objects are recognized with CNN models trained for detecting specific objects such as the known components being inspected and further teaches that defects are identified and classified (paragraphs [0052], [0055] and [0069]).
Therefore it would have been obvious to one of ordinary skill in the art before time of filing to use component image analysis and object recognition taught by Wang in combination with the image recognition of Domke in order recognize and identify known components being inspected.
With regard to claim 2, Domke discloses the inspection system as recited in claim 1, wherein the images are of the feature of the component at the determined location (paragraphs [0032]-[0034], [0038]-[0039] and [0042], component parts of imaged for inspection).
With regard to claim 3, Domke discloses the inspection system as recited in claim 1, wherein the controller is further configured to identify a target image of the component and to use the identified target image to determine the location of the captured images and to generate the graphic images associated with the determined location (paragraphs [0009], [0049] and [0071], The location of the borescope tip can be displayed as an overlay in addition to measurement and annotation overlays. The current location can be identified in relation to an object being inspected).
Domke does not explicitly disclose the added amended limitations.
Wang discloses identify a target image of the component and surrounding features within the captured images, and to use the identified target image and the surrounding features to determine the location by correlating the target image and the surrounding features to the stored model of the component of the captured images and to generate the graphic images associated with the determined location (paragraphs [0052], [0055] and [0069], Wang discloses a similar borescope inspection method and device and teaches that object detection is performed and that the object detection may be performed with convolutional neural networks trained for specific parts of the engine being inspected, meaning that the detected objects are recognized with CNN models trained for detecting specific objects such as the known components being inspected and further teaches that defects are identified and classified according to the specific location or section of the engine being analyzed. Target image of components and surrounding features is interpreted as the specific component and specific related component defects).
Therefore it would have been obvious to one of ordinary skill in the art before time of filing to use component image analysis and object recognition taught by Wang in combination with the image recognition of Domke in order recognize and identify known components being inspected.
With regard to claim 4, Domke discloses the inspection system as recited in claim 3, wherein the target image comprises at least one of a bar code, QR code, or a defined pattern on the component and the controller is further programmed to associate the target image with the location of the captured image (paragraph [0071], The location can be determined by imaging component parts as well as bar code identifiers or recognition logic upon a captured image portion).
Domke doesn’t explicitly disclose the newly amended limitations.
Wang discloses the step to associate the target image with the location of the captured image by correlating the target image to the stored model of the component (paragraphs [0052], [0055] and [0069], Wang discloses a similar borescope inspection method and device and teaches that object detection is performed and that the object detection may be performed with convolutional neural networks trained for specific parts of the engine being inspected, meaning that the detected objects are recognized with CNN models trained for detecting specific objects such as the known components being inspected and further teaches that defects are identified and classified according to the specific location or section of the engine being analyzed. The target or object recognized image is correlated to the stored model of the component by identifying the trained CNN for a specific area and recognizing the object and classifying potential component defects).
Therefore it would have been obvious to one of ordinary skill in the art before time of filing to use component image analysis and object recognition taught by Wang in combination with the image recognition of Domke in order recognize and identify known components being inspected.
With regard to claim 5, Domke discloses the inspection system as recited in claim 3, wherein the controller is configured to continuously update the determination of the location of the camera system and captured images based on identification of one or more target images (paragraphs [0071]-[0077], The identification of the location of the borescope is performed in an ongoing and updating manner as the borescope changes position and orientation).
With regard to claim 6, Domke discloses the inspection system as recited in claim 3, wherein the controller is configured to generate a virtual button associated with features of the component and to superimpose the virtual button on the real-time display and the system further includes a means of actuating the virtual button (paragraph [0049], A variety of overlays including menus are disclosed. A menu with clickable options is interpreted as a virtual button that may be clicked or allow user interaction).
With regard to claim 7, Domke discloses the inspection system as recited in claim 3, wherein the graphic image comprises at least one of a label, tag, arrow, or overlay that corresponds to a feature of the component (paragraph [0056], Overlay data examples include arrow pointers, crosses, geometric shapes, etc.).
With regard to claim 8, Domke discloses the inspection system as recited in claim 3, wherein the graphic image comprises inspection directions prompting the inspection of features of the component (paragraphs [0060] and [0067], The user is guided through the inspection with instructions and supplemental data).
With regard to claim 9, Domke discloses the inspection system as recited in claim 8, wherein the inspection directions include a visual prompt directing movement of the camera system from the location determined based on captured images to another location or orientation for another feature of the component (paragraphs [0030], [0049], [0060],The user of the borescope camera is provided guidance on how and where to move the camera to perform inspection in a series of guided steps).
With regard to claim 10, Domke discloses the inspection system as recited in claim 3, wherein the graphic image comprises a visual representation of a geometry of a feature of the component superimposed on the real-time display over a corresponding feature of the component (paragraph [0049], real time overlays are provided for the inspection images including measurement overlays which is considered a visual representation of geometry of a feature).
With regard to claim 11, Domke discloses the inspection system as recited in claim 3, wherein the controller is further configured to generate one of a still image, an audio message, a video for viewing on the display in response to identification of the target image (Audio, paragraph [0056]; Video and images, paragraphs [0039], [0041], [0044] and [0049]-[0051]).
With regard to claim 12, Domke discloses the inspection system as recited in claim 1, wherein the camera system comprises a borescope configured to capture images of internal features of the component (paragraph [0032], the borescope is inserted into a plurality of borescope ports and other location of the turbo machinery to capture images of internal features).
With regard to claim 13, Domke discloses the inspection system as recited in claim 1, wherein the display comprises at least one of a portable display or a headset configured to generate images in real-time from the camera system and to superimpose the graphic images generated by the controller (paragraphs [0025] and [0049], A tablet portable display is disclosed for displaying the inspection images in real time with overlay information).
With regard to claim 14, Domke discloses the inspection system as recited in claim 1, wherein the controller is further programmed to generate documentation of inspection findings based on the captured images (Fig. 5, report 158,159 and paragraphs [0045]-[0047], The inspection images are used to analyze and document a report of the inspection).
With regard to claim 15, Domke discloses a method of inspecting a turbine engine comprising:
capturing images of features of a component of the turbine engine with a movable camera system (paragraphs [0025], [0032], [0049], A borescope camera system is used to images of internal components of a turbine);
determining a location of the movable camera system and a captured image of a feature of the component with a processor (paragraphs [0009], [0028], and [0036], processor is disclosed to perform the detection logic programming) in communication with a memory storing a model of the component (paragraphs [0028], [0036], [0045], [0059], Memory is disclosed for storing images and detections of the images)based on features of the component within the captured image (paragraph [0049], a borescope tip map can be displayed and overlaid on the display image to give an approximation of the location of the borescope tip as a means to guide the operator);
generating a graphic image associated with a feature of the component at the determined location (paragraph [0049], Multiple overlays are generated for the inspection image); and
generating a real-time display of the component on a display device utilizing the captured images and the generated graphic image superimposed over the captured images of the component (paragraphs [0025]-[0026] and [0049], A display is connected to the borescope camera and capable of overlaying certain data onto the image for more informative view).
Domke discloses performing image recognition (paragraphs [0071], [0075]) in order to identify the location of the probe and to inspect objects. Image recognition implies an analysis and comparison to a known image of the object, however Domke does not explicitly recite the added limitation of by performing an image analysis on features appearing within the captured images and correlating the features to the stored model of the component
Wang discloses a similar borescope inspection method and device and teaches that object detection is performed and that the object detection may be performed with convolutional neural networks trained for specific parts of the engine being inspected, meaning that the detected objects are recognized with CNN models trained for detecting specific objects such as the known components being inspected and further teaches that defects are identified and classified (paragraphs [0052], [0055] and [0069]).
Therefore it would have been obvious to one of ordinary skill in the art before time of filing to use component image analysis and object recognition taught by Wang in combination with the image recognition of Domke in order recognize and identify known components being inspected.
With regard to claim 16, Domke discloses the method as recited in claim 15, further comprising determining a location of the movable camera system within the component part based on identification of a target image within the captured images of the component and using the identified target image to determine the location of the captured images and to trigger generation of graphic images corresponding to the determined location (paragraph [0049], a borescope tip map can be displayed and overlaid on the display image to give an approximation of the location of the borescope tip as a means to guide the operator. The overlays are generated in response to specific recognized image content such as object identification overlays).
Domke does not explicitly disclose the added amended limitations.
Wang discloses determining a location of the movable camera system within the component part based on identification of a target image and surrounding features within the captured images of the component and correlating the features to the stored model of the component to determine the location of the captured images and to trigger generation of graphic images corresponding to the determined location. (paragraphs [0052], [0055]-[0057] and [0069], Wang discloses a similar borescope inspection method and device and teaches that object detection is performed and that the object detection may be performed with convolutional neural networks trained for specific parts of the engine being inspected, meaning that the detected objects are recognized with CNN models trained for detecting specific objects such as the known components being inspected and further teaches that defects are identified and classified according to the specific location or section of the engine being analyzed. Target image of components and surrounding features is interpreted as the specific component and specific related component defects. The displayed images can be augmented with a defect graphic overlay for example).
Therefore it would have been obvious to one of ordinary skill in the art before time of filing to use component image analysis and object recognition taught by Wang in combination with the image recognition of Domke in order recognize and identify known components being inspected.
With regard to claim 17, Domke discloses the method as recited in claim 16, wherein the target image comprises at least one of a bar code, QR code, or a defined pattern affixed to the component (paragraph [0071], The location can be determined by imaging component parts as well as bar code identifiers or recognition logic upon a captured image portion).
Domke doesn’t explicitly disclose the newly amended limitations.
Wang discloses the step of determining the location of the captured images further comprises correlating the target image to the stored model of the component. (paragraphs [0052], [0055] and [0069], Wang discloses a similar borescope inspection method and device and teaches that object detection is performed and that the object detection may be performed with convolutional neural networks trained for specific parts of the engine being inspected, meaning that the detected objects are recognized with CNN models trained for detecting specific objects such as the known components being inspected and further teaches that defects are identified and classified according to the specific location or section of the engine being analyzed. The target or object recognized image is correlated to the stored model of the component by identifying the trained CNN for a specific area and recognizing the object and classifying potential component defects).
Therefore it would have been obvious to one of ordinary skill in the art before time of filing to use component image analysis and object recognition taught by Wang in combination with the image recognition of Domke in order recognize and identify known components being inspected.
With regard to claim 18, Domke discloses the method as recited in claim 17, further comprising generating a virtual button associated with features of the component, superimposing the virtual button on the real-time display, and triggering a predefined action in response to triggering of the virtual button (paragraph [0049], A variety of overlays including menus are disclosed. A menu with clickable options is interpreted as a virtual button that may be clicked or allow user interaction).
With regard to claim 19, Domke discloses the method as recited in claim 16, further comprising identifying a feature of the component based on captured images and generating inspection directions corresponding to the identified feature of the component (paragraphs [0060], [0067] and [0075], The user is guided through the inspection with instructions and supplemental data based on recognition of a location within the object of inspection).
With regard to claim 20, Domke discloses the method as recited in claim 19, further comprising automatically generating documentation of inspection findings based on the captured images upon completion of the generated inspection directions (Fig. 5, report 158,159 and paragraphs [0045]-[0047], The inspection images are used to analyze and document a report of the inspection).
FINAL REJECTION
Applicant’s amendment necessitated the new grounds of rejection presented in the Office Action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Contact Information
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WESLEY J TUCKER whose telephone number is (571)272-7427. The examiner can normally be reached 9AM-5PM Monday-Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JOHN VILLECCO can be reached at 571-272-7319. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WESLEY J TUCKER/Primary Examiner, Art Unit 2661
Prosecution Insights