DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Received 07/07/2026
Claim(s) 1, 5-14, and 18-30 is/are pending.
Claim(s) 26 and 30 has/have been amended.
Claim(s) 2-4 and 15-17 has/have been cancelled.
The 35 U.S.C § 103 rejection to claim(s) 26-30 have been fully considered in view of the amendments received on 07/07/2026 and are fully addressed in the prior art rejection below.
Claim(s) 1, 5-14, and 18-25 are allowed.
Response to Arguments
Received 07/07/2026
Regarding independent claim(s) 26 and 30:
Applicant’s arguments (Remarks, Page 10: ¶ 2 to Page 11: ¶ 3), filed 07/07/2026, with respect to the rejection(s) of claim(s) 26 under 35 U.S.C § 103 have been fully considered and are persuasive. Wherein, Troy et al. (US PGPUB No. 20170243399 A1) fails to explicitly disclose the defect is identified prior to retrieving the stored image-overlaid model. Therefore, the rejection has been withdrawn, necessitated by Applicant's amendments. However, upon further consideration, a new ground(s) of rejection is made in view of Troy et al., in view of Scott et al. (US PGPUB No. 20230073587 A1), in view of Dann et al. (US PGPUB No. 20120300984 A1), and further in view of Froom et al. (US PGPUB No. 20130261876 A1).
Applicant’s arguments (Remarks, Page 12: ¶ 2 to Page 14: ¶ 1), filed 07/07/2026, with respect to the rejection(s) of claim(s) 30 under 35 U.S.C § 103 have been fully considered and are persuasive. Wherein, Scott et al. fails to disclose the three-dimension computer model is identical in each of the image-overlaid models. Therefore, the rejection has been withdrawn, necessitated by Applicant's amendments. However, upon further consideration, a new ground(s) of rejection is made in view of Scott et al., in view of Chen et al. (US PGPUB No. 20240420454 A1), in view of Dann et al., and further in view of Park et al. (US Patent No. 6149435 A1).
Regarding dependent claim(s) 27-29:
Applicant’s arguments (Remarks, Page 11: ¶ 4), filed 07/07/2026, with respect to the rejection(s) of claim(s) 27-29 under 35 U.S.C § 103 have been fully considered and are persuasive due the dependency upon claims 26 respectively. Therefore, the rejection has been withdrawn, necessitated by Applicant's amendments. However, upon further consideration, a new ground(s) of rejection is made in view of the prior art as mentioned above.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 26-29 is/are rejected under 35 U.S.C. 103 as being unpatentable over Troy et al., US PGPUB No. 20170243399 A1, hereinafter Troy, in view of Scott et al., US PGPUB No. 20230073587 A1, hereinafter Scott, in view of Dann et al., US PGPUB No. 20120300984 A1, hereinafter Dann, and further in view of Froom et al., US PGPUB No. 20130261876 A1, hereinafter Froom.
Regarding claim 26, Troy discloses a method for retroactively auditing a condition of a large-scale manufacturing product (Troy; a method for retroactively auditing (corresponding to a 3D visualization application accessing a 3D model) a condition of a large-scale manufacturing product [¶ 0024 and ¶ 0067-0068]), the method comprising:
capturing one or more camera images of the large-scale manufacturing product, each at a respective first point in perspective, each camera image depicting a respective captured surface region of the large-scale manufacturing product (Troy; capturing one or more camera images of the large-scale manufacturing product each camera image depicting a respective captured surface region of the large-scale manufacturing product each at a respective 1st point of view / perspective [¶ 0027 and ¶ 0031-0032]);
providing a three-dimensional computer model of the large-scale manufacturing product independent from the one or more camera images (Troy; providing a 3D computer model of the large-scale manufacturing product independent from the one or more camera images [¶ 0033 and ¶ 0036]; wherein, the 3D model is from a DB in relation with the 3D visualization application is independent for the data/images received from the digital camera [¶ 0062]);
an image-overlaid model comprising the three-dimensional computer model of the large-scale manufacturing product and each of the one or more camera images overlaid onto the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region (Troy; an image-overlaid model comprising the 3D computer model of the large-scale manufacturing product and each of the one or more camera images overlaid onto the 3D computer model in a respective position [¶ 0031-0033, ¶ 0037, and ¶ 0039]; moreover, an overlay according to a viewpoint [¶ 0034-0035]);
at a second point in perspective after each respective first point in perspective, determining a region of interest of the large scale manufacturing product (Troy; determining a ROI of the large scale manufacturing product at a 2nd point of view / perspective after each respective first point of view / perspective [¶ 0036-0037 and ¶ 0052]; wherein, interaction with a model correlates to viewpoint is manipulated [¶ 0060]; moreover, computer viewpoint and 3D models displayed in the virtual environment produces an approximate viewpoint based on photograph of the target object [¶ 0033-0034]); and
displaying a view of the image-overlaid model that includes the captured camera images depicting a captured surface region including said the determined region of interest of the large-scale manufacturing product (Troy; displaying a view of the image-overlaid model that includes the captured camera images depicting a captured surface region including said the determined ROI of the large-scale manufacturing product [¶ 0060-0063 and ¶ 0065]; moreover, manipulating the viewpoint of a 3D model [¶ 0060]).
Troy fails to disclose captured surface regions in a condition free of defects at the first point in time;
generating an image-overlaid model comprising the three-dimensional computer model;
storing the image-overlaid model;
at a second point in time after each respective first point in time; and
in response to identifying the defect, retrieving the stored image-overlaid model and displaying a view of the image-overlaid model that includes the captured camera images depicting a captured surface region including said the determined region of interest of the large-scale manufacturing product, thereby enabling verification that the region of interest was in the condition free of defects at the first potin in time.
wherein the defect is identified during a visual inspection of the large-scale manufacturing product at the second point in time and wherein the defect is identified prior to retrieving the stored image-overlaid model for purposes of verification the region of interest was in the condition free of defects at the first point in time.
However, Scott teaches capturing one or more camera images of the large-scale manufacturing product, each at a respective first point in time (Scott; capturing one or more camera images of the large-scale manufacturing product each at a respective 1st point in time [¶ 0035-0036 and ¶ 0040]), each camera image depicting a respective captured surface region of the large-scale manufacturing product (Scott; each camera image depicting a respective captured surface region of the large-scale manufacturing product [¶ 0018-0019 and ¶ 0035-0036]), wherein the one or more camera images document the respective captured surface regions in a condition free of defects at the first point in time (Scott; the one or more camera images document the respective captured surface regions in a condition implicitly free of defects (given that the stored baseline image is utilized for comparisons) at the 1st point in time (i.e. a scan at a different time associated with a history) [¶ 0037-0038 and ¶ 0040]; moreover, comparing the 3D image record with one or more of a stored design model for the object or a stored baseline image record of the object [¶ 0049]; such that, generating findings data based on the comparing, where the findings data is indicative of a discrepancy identified between the 3D image record and the one or more of the stored design model for the object or the stored baseline image record of the object [¶ 0050, ¶ 0052, and ¶ 0066]);
providing a three-dimensional computer model of the large-scale manufacturing product independent from the one or more camera images (Scott; providing a 3D computer model of the large-scale manufacturing product independent from the one or more camera images [¶ 0035-0037]; wherein, design models or baseline scans are different than currently captured data [¶ 0017-0018]);
generating an image model comprising the three-dimensional computer model of the large-scale manufacturing product and each of the one or more camera images form the three-dimensional computer model (Scott; generating an image model comprising the 3D computer model of the large-scale manufacturing product and each of the one or more camera images form the 3D computer model [¶ 0015, ¶ 0024, and ¶ 0035-0037]) in a respective position, size, and orientation corresponding to the respective captured surface region (Scott; in an implicit respective position, size, and orientation (given alignment techniques) corresponding to the respective captured surface region [¶ 0017 and ¶ 0021-0022]);
storing the image model (Scott; storing the image model [¶ 0037-0038 and ¶ 0040]; additionally, generating 3D image records [¶ 0017 and ¶ 0021]);
at a second point in time after each respective first point in time, identifying a defect in determining a region of interest of the large scale manufacturing product (Scott; identifying a defect in determining a ROI (i.e. compared scan surface) of the large scale manufacturing product at an implicit 2nd point in time after each respective 1st point in time (given scanned history and lifecycle) [¶ 0036-0038 and ¶ 0040]; moreover, changes over time [id.]; in other words, a 1st point in time corresponds to a recorded baseline image model and 2nd point in time corresponds to a updated scan image model [¶ 0015]; wherein, image capturing of an object occurs at any stage of manufacturing [¶ 0018]); and
in response to identifying the defect, retrieving the stored image model and displaying a view of the image model that includes the captured camera images depicting a captured surface region including said the determined region of interest of the large-scale manufacturing product (Scott; retrieving the stored image model and displaying a view of the image model that includes the captured camera images depicting a captured surface region including said the determined ROI (i.e. compared scan surface) of the large-scale manufacturing product in response to identifying the defect [¶ 0036-0038]; moreover, information for the 3D image record is stored along with the 3D image record and used to track the object during manufacturing or object lifecycle [¶ 0040]; wherein, a repository is also operable to store a plurality of sets of findings data, each set of findings data corresponding to one or more of the plurality of 3D image records for the object, and the findings data indicates any discrepancies identified between the 3D image record and the stored design model for the object or the stored baseline image record of the object used for the comparison [id.]; additionally, the 3D image records for an object can provide a history for the object, including changes over time, etc., and be used to identify and quantify changes, defects, and/or anomalies [id.]), thereby enabling verification that the region of interest condition at the first potin in time (Scott; thereby enabling verification that the ROI (i.e. compared scan surface) condition at the first potin in time (corresponding to a baseline and/or tracked history) [¶ 00037-0038 and ¶ 0040]);
wherein the defect is identified during a visual inspection of the large-scale manufacturing product at the second point in time (Scott; the defect is identified during a visual inspection of the large-scale manufacturing product (i.e. airplane/fuselage) at the 2nd point in time (i.e. current capture, sch that a 1st point in time corresponds to a previous capture (i.e. baseline)) [¶ 0014-0015, ¶ 0017, and ¶ 0040]; additionally, comparison of the 3D image record with one or more of a stored design model (baseline) [¶ 0036-0038]).
Troy and Scott are considered to be analogous art because both pertain to generating and/or managing data in relation with sensor data and/or captured images, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Troy, to incorporate capturing one or more camera images of the large-scale manufacturing product, each at a respective first point in time, each camera image depicting a respective captured surface region of the large-scale manufacturing product, wherein the one or more camera images document the respective captured surface regions in a condition free of defects at the first point in time; providing a three-dimensional computer model of the large-scale manufacturing product independent from the one or more camera images; generating an image model comprising the three-dimensional computer model of the large-scale manufacturing product and each of the one or more camera images form the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region; storing the image model; at a second point in time after each respective first point in time, identifying a defect in determining a region of interest of the large scale manufacturing product; in response to identifying the defect, retrieving the stored image model and displaying a view of the image model that includes the captured camera images depicting a captured surface region including said the determined region of interest of the large-scale manufacturing product, thereby enabling verification that the region of interest condition at the first potin in time; and wherein the defect is identified during a visual inspection of the large-scale manufacturing product at the second point in time (as taught by Scott), in order to provide an improved automated and consistent imaging capture and comparison to design specifications or baseline image recordings (Scott; [¶ 0002-0004 and ¶ 0014-0015]).
Troy as modified by Scott fails to disclose generating an image-overlaid model comprising the three-dimensional computer model of the large-scale manufacturing product and each of the one or more camera images overlaid onto the three-dimensional computer model;
enabling verification that the region of interest was in the condition free of defects at the first potin in time; and
wherein the defect is identified prior to retrieving the stored image-overlaid model for purposes of verification the region of interest was in the condition free of defects at the first point in time.
However, Dann teaches generating an image-overlaid model comprising the three-dimensional computer model of the large-scale manufacturing product and each of the one or more camera images overlaid onto the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region (Dann; generating an image-overlaid model comprising the 3D computer model of the large-scale manufacturing product and each of the one or more camera images overlaid onto the 3D computer model in an implicit respective position, size, and orientation (given alignment techniques) corresponding to the respective captured surface region [¶ 0011-0015]; moreover, 3D virtual design model of an object [¶ 0006 and ¶ 0018]);
storing the image-overlaid model (Dann; storing the image-overlaid model [¶ 0035-0037 and ¶ 0044]); and
in response to identifying the defect, retrieving the stored image-overlaid model and displaying a view of the image-overlaid model that includes the captured camera images depicting a captured surface region including said the determined region of interest of the large-scale manufacturing product (Dann; retrieving the stored image model and displaying a view of the image model that includes the captured camera images depicting a captured surface region including said the determined ROI of the large-scale manufacturing product in response to identifying the defect [¶ 0036-0037 and ¶ 0040-0042]; moreover, with the damage layer and the real image layer laid over the model generated image [id. and ¶ 0046]), thereby enabling verification that the region of interest was in the condition free of defects at the first potin in time (Dann; thereby enabling verification that the region of interest was in the condition free of defects at the 1st potin in time (corresponding to a recorded history) [¶ 0044 and ¶ 0046-0047]; wherein a condition of being defect free is desirable given the recorded history [¶ 0041-0044], such that information may include one or more of the location, history, repair status, severity etc. of each recorded damage [¶ 0037] via information stored within a DB [¶ 0045-0047]).
Troy in view of Scott and Dann are considered to be analogous art because they pertain to generating and/or managing data in relation with sensor data and/or captured images, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Troy as modified by Scott, to incorporate generating an image-overlaid model comprising the three-dimensional computer model of the large-scale manufacturing product and each of the one or more camera images overlaid onto the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region; storing the image-overlaid model; and in response to identifying the defect, retrieving the stored image-overlaid model and displaying a view of the image-overlaid model that includes the captured camera images depicting a captured surface region including said the determined region of interest of the large-scale manufacturing product, thereby enabling verification that the region of interest was in the condition free of defects at the first potin in time (as taught by Dann), in order to provide quick and accurate identification of one or more points of interest in combination with a reduction in operational disruptions (Dann; [¶ 0006-0007, ¶ 0021, and ¶ 0025]).
Troy as modified by Scott and Dann fails to explicitly disclose wherein the defect is identified prior to retrieving the stored image-overlaid model for purposes of verification the region of interest was in the condition free of defects at the first point in time.
However Froom teaches wherein the defect is identified during a visual inspection of the large-scale manufacturing product at the second point in time and wherein the defect is identified prior to retrieving the stored image-overlaid model for purposes of verification the region of interest was in the condition free of defects at the first point in time (Froom; the defect is identified during a visual inspection of the large-scale manufacturing product [¶ 0064 and ¶ 0066-0067] at the 2nd point in time and wherein the defect is identified prior to retrieving the stored image-overlaid model (i.e. baseline) for purposes of verification the region of interest was in the condition free of defects at the 1st point in time [¶ 0072-0073 and ¶ 0131-0133]; wherein, modeling airplanes corresponds to image-overlaid model [¶ 0096-0097 and ¶ 0099]; moreover, a defect report and/or baseline obtained is archived so that at a later time, it is possible to conduct a trend analysis or conduct a baseline comparison [¶ 0101-0102 and ¶ 0143-0144]; additionally, future structural defects and deficiencies may be formulated and predicted [¶ 0074] in relation with projecting needed maintenance [¶ 0100]).
Troy in view of Scott and Dann and Froom are considered to be analogous art because they pertain to generating and/or managing data in relation with sensor data and/or captured images, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Troy as modified by Scott and Dann, to incorporate wherein the defect is identified during a visual inspection of the large-scale manufacturing product at the second point in time and wherein the defect is identified prior to retrieving the stored image-overlaid model for purposes of verification the region of interest was in the condition free of defects at the first point in time (as taught by Froom), in order to provide non-destructive inspection and testing of airplanes (Froom; [¶ 0003-0005 and ¶ 0012-0014]).
Regarding claim 27, Troy in view of Scott, Dann, and Froom further discloses the method of claim 26, wherein the large-scale manufacturing product is an airframe (Troy; the large-scale manufacturing product is an airframe/airplane [¶ 0024 and ¶ 0027]).
Scott further teaches the large-scale manufacturing product is an airframe (Scott; the large-scale manufacturing product is airframe (aircraft fuselage) [¶ 0018-0019]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Troy as modified by Scott, Dann, Froom, to incorporate the large-scale manufacturing product is an airframe (as taught by Scott), in order to provide an improved automated and consistent imaging capture and comparison to design specifications or baseline image recordings (Scott; [¶ 0002-0004 and ¶ 0014-0015]).
Regarding claim 28, Troy in view of Scott, Dann, and Froom further discloses the method of claim 26, wherein the large-scale manufacturing product is a fuselage (Scott; the large-scale manufacturing product is a fuselage [¶ 0018-0019]).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Troy as modified by Scott, Dann, and Froom, to incorporate the large-scale manufacturing product is a fuselage (as taught by Scott), in order to provide an improved automated and consistent imaging capture and comparison to design specifications or baseline image recordings (Scott; [¶ 0002-0004 and ¶ 0014-0015]).
Regarding claim 29, Troy in view of Scott, Dann, and Froom further discloses the method of claim 26, wherein the large-scale manufacturing product is a fuselage interior (Scott; the large-scale manufacturing product is a fuselage interior [¶ 0018 and ¶ 0028], as illustrated within Fig. 1D).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Troy as modified by Scott, Dann, and Froom, to incorporate the large-scale manufacturing product is a fuselage interior (as taught by Scott), in order to provide an improved automated and consistent imaging capture and comparison to design specifications or baseline image recordings (Scott; [¶ 0002-0004 and ¶ 0014-0015]).
Claim(s) 30 is/are rejected under 35 U.S.C. 103 as being unpatentable over Scott, in view of Chen et al., US PGPUB No. 20240420454 A1, hereinafter Chen, in view of Dann, and further in view of Park et al., US Patent No. 6149435 A1, hereinafter Park.
Regarding claim 30, Scott discloses a method of identifying foreign object debris (FOD) on a type of large-scale manufacturing product (Scott; a method of identifying FOD (i.e. discrepancies) on a type of large-scale manufacturing product [¶ 0014-0015 and ¶ 0017-0018]; moreover, identifying discrepancies [¶ 0038, ¶ 0040, ¶ 0050, and ¶ 0052]), the method comprising:
capturing a plurality of camera images of the respective previous unit (Scott; the method [as addressed above] comprises capturing a plurality of camera images of the respective previous unit (corresponding to stored image data of an object) [¶ 0035-0038]; moreover, information for the 3D image record is stored along with the 3D image record and used to track the object during manufacturing or object lifecycle [¶ 0040]; wherein, a repository is also operable to store a plurality of sets of findings data, each set of findings data corresponding to one or more of the plurality of 3D image records for the object, and the findings data indicates any discrepancies identified between the 3D image record and the stored design model for the object or the stored baseline image record of the object used for the comparison [id.]; additionally, the 3D image records for an object can provide a history for the object, including changes over time, etc., and be used to identify and quantify changes, defects, and/or anomalies [id.]), each camera image depicting a respective captured surface region of the respective previous unit (Scott; each camera image depicting a respective captured surface region of the respective previous unit [¶ 0037-0038 and ¶ 0040]; moreover, comparing the 3D image record with one or more of a stored design model for the object or a stored baseline image record of the object [¶ 0049] associated with captured images [¶ 0035-0036]; such that, generating findings data based on the comparing, where the findings data is indicative of a discrepancy identified between the 3D image record and the one or more of the stored design model for the object or the stored baseline image record of the object [¶ 0050, ¶ 0052, and ¶ 0066]); and
generating a respective image-overlaid model by overlaying each of the plurality of camera images onto the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region (Scott; the method [as addressed above] comprises generating an image model comprising the 3D computer model of the large-scale manufacturing product and each of the one or more camera images form the 3D computer model [¶ 0015, ¶ 0024, and ¶ 0035-0037] in an implicit respective position, size, and orientation (given alignment techniques) corresponding to the respective captured surface region [¶ 0017 and ¶ 0021-0022]);
training a machine learning model for FOD-identification based on the plurality of image models of previous units of large-scale manufacturing product (Scott; the method [as addressed above] comprises training a ML model [¶ 0017 and ¶ 0060] for FOD-identification (i.e. identifying deficiencies) based on a plurality of image models of previous units (i.e. recorded/stored image data/models) of large-scale manufacturing products [¶ 0037-0038, ¶ 0040, ¶ 0049-0050, and ¶ 0052]; wherein, an image model is implicit (given image alignment and/or stitching in the process of form the model) [¶ 0021-0022, ¶ 0036-0037, and ¶ 0048-0049]);
capturing new camera images of a new unit of the large-scale manufacturing product (Scott; the method [as addressed above] comprises capturing new camera images of a new unit of the large-scale manufacturing product [¶ 0034-0036 and ¶ 0040]; wherein, several sensors provide image data corresponding to one or more images correlated to 3D imaging [¶ 0047-0049 and ¶ 0052]; wherein new or updated images are obtained at different stages in a lifecycle [¶ 0015-0018, ¶ 0035-0036, and ¶ 0040]), each new camera image depicting a respective captured surface region of the new unit (Scott; each new/updated camera image depicting a respective captured surface region of the new/updated unit [¶ 0015-0017]; moreover, image capturing scenarios [¶ 0018] in relation with 3D imaging involving multiple images [¶ 0015-0017 and ¶ 0020-0021] in relation with stages of a manufacture lifecycle [¶ 0037-0038, ¶ 0040, ¶ 0049, and ¶ 0052]);
generating a new image model comprising the three-dimensional computer model and each of the new camera images form the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region (Scott; the method [as addressed above] comprises generating a new/updated image model comprising the 3D computer model [¶ 0036-0037 and ¶ 0048-0049] and each of the new/updated camera images form the 3D computer model in a respective position, size, and orientation (given image alignment and/or stitching related to the process of forming the model) corresponding to the respective captured surface region [¶ 0021-0022]; moreover, images in relation with CAD/CAM model(s) [¶ 0037 and ¶ 0049]; and moreover, converting captured image into a 3D image record (i.e. model) [¶ 0015, ¶ 0040, and ¶ 0052]); and
using the machine learning model for FOD-identification to identify FOD based on the new camera images (Scott; using the ML model [as addressed above] for FOD-identification (i.e. identifying deficiencies) to identify FOD based on the new/updated camera images [¶ 0037-0038, ¶ 0040, ¶ 0049-0050, and ¶ 0052]; wherein, inspection can be used at any point in relation with identifying deficiencies [¶ 0015 and ¶ 0017]).
Scott fails to disclose for each of a plurality of previous units of said type of large-scale manufacturing product:
a computer model of the type of large-scale manufacturing product; and
generating a respective image-overlaid model by overlaying each of the plurality of camera images onto the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region;
training of a type of model;
wherein each image-overlaid model comprises camera images of the respective unit overlaid on the three-dimensional model of the type of large-scale manufacturing product; and
wherein the three-dimensional computer model is identical in each of the image-overlaid models for the previous units used during said training the machine learning model and in said new image-overlaid model.
However, Chen teaches for each of a plurality of previous units of said type of large-scale manufacturing product (Chen; for each of a plurality of previous units (i.e. training data of an object) of said type of large-scale manufacturing product [¶ 0057-0060], as illustrated within Fig. 3):
capturing a plurality of camera images of the respective previous unit (Chen; capturing a plurality of camera images (via one or more detectors) of the respective previous unit (i.e. training data of an object) [¶ 0057-0060]; wherein, camera images corresponds to image dataset [¶ 0035-0036]; additionally, ground truth data [¶ 0049 and ¶ 0061]), each camera image depicting a respective captured surface region of the respective previous unit (Chen; each camera image depicting a respective captured surface region [¶ 0033 and ¶ 0035-0036] of the respective previous unit (i.e. training data of an object) [¶ 0059-0060]; wherein, Fig. 1 illustrates, a capture surface region (or ROI) [¶ 0039-0040] associated with one or more objects of interest [¶ 0038, ¶ 0044-0045, and ¶ 0051-0052]);
training a machine learning model for FOD-identification based on a plurality of models of previous units of said type of large-scale manufacturing products (Chen; training a machine learning model for FOD-identification based on a plurality of models of previous units of said type of large-scale manufacturing products [¶ 0057-0058, ¶ 0060, and ¶ 0062]);
providing a computer model of the type of large-scale manufacturing product independent from the camera images (Chen; providing a computer model of the type of large-scale manufacturing product independent from the camera images [¶ 0040 and ¶ 0062]; moreover, different configurations or classes of an object [¶ 0043-0044 and ¶ 0051-0052] associated with a trained detector [¶ 0054-0056]);
using the machine learning model for FOD-identification to identify FOD based on the new camera images (Chen; using the machine learning model for FOD-identification to identify FOD based on the implicit new camera images (given the training of the detector to be used in a non-training environment) [¶ 0002-0003, ¶ 0060, and ¶ 0062]); and
wherein the computer model is similar in each of the image models for the previous units used during said training the machine learning model and in said new image model (Chen; the computer model is (verified as) similar in each of the image models for the previous units (i.e. training data) used during said training the ML model [¶ 0049-0050, ¶ 0053, and ¶ 0059] and in said new image model [¶ 0060-0061]; wherein, training is in relation with a ML model [¶ 0002-0003]; wherein, training images are require to having an object in order to be considered (i.e. the object is required to be identified in the image to be used in training) [¶ 0059]; and wherein, the model corresponds to the computerized object [¶ 0035-0036]; in other words, 1st and 2nd training datasets of a captured object [¶ 0024] are determined using positive images corresponding to images from the two datasets that both comprise the object, implicitly meaning that the object is the same in both [¶ 0025-0026 and ¶ 0046]).
Scott and Chen are considered to be analogous art because both pertain to generating and/or managing data in relation with sensor data and/or captured images, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Scott, to incorporate for each of a plurality of previous units of said type of large-scale manufacturing product: capturing a plurality of camera images of the respective previous unit, each camera image depicting a respective captured surface region of the respective previous unit; training a machine learning model for FOD-identification based on a plurality of models of previous units of said type of large-scale manufacturing products; providing a computer model of the type of large-scale manufacturing product independent from the camera images; using the machine learning model for FOD-identification to identify FOD based on the new camera images; and wherein the computer model is similar in each of the image models for the previous units used during said training the machine learning model and in said new image model (as taught by Chen), in order to provide improved machine learning predictions using training techniques (Chen; [¶ 0002-0004]).
Scott as modified by Chen fails to disclose generating a respective image-overlaid model by overlaying each of the plurality of camera images onto the three-dimensional computer model; and
wherein the three-dimensional computer model is identical in each of the image-overlaid models and in said new image-overlaid model.
However, Dann teaches generating a respective image-overlaid model by overlaying each of the plurality of camera images onto the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region (Dann; generating a respective image-overlaid model by overlaying each of the plurality of camera images onto the 3D computer model in an implicit respective position, size, and orientation (given alignment techniques) corresponding to the respective captured surface region [¶ 0011-0015]; moreover, 3D virtual design model of an object [¶ 0006 and ¶ 0018]).
Scott in view of Chen and Dann are considered to be analogous art because they pertain to generating and/or managing data in relation with sensor data and/or captured images, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Scott as modified by Chen, to incorporate generating a respective image-overlaid model by overlaying each of the plurality of camera images onto the three-dimensional computer model in a respective position, size, and orientation corresponding to the respective captured surface region (as taught by Dann), in order to provide quick and accurate identification of one or more points of interest in combination with a reduction in operational disruptions (Dann; [¶ 0006-0007, ¶ 0021, and ¶ 0025]).
Scott as modified by Chen and Dann fail to disclose wherein the three-dimensional computer model is identical in each of the image-overlaid models and in said new image-overlaid model.
However, Park teaches the three-dimensional computer model is identical in each of the image-overlaid models for the previous units used during said training and in said new image-overlaid model (Park; the 3D computer model is identical in each of the image-overlaid models for the previous units used during said training (i.e. simulation) and in said new image-overlaid model [Col. 1, lines 50-65 and Col. 2, lines 11-31 and lines 46-57]; wherein, training of a human or a machine can be comparable).
Scott in view of Chen and Dann and Park are considered to be analogous art because they pertain to generating and/or managing data in relation with sensor data and/or captured images, wherein one or more computerized units are utilized in order to produce a visualization effect.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing of the claimed invention was made to modify Scott as modified by Chen and Dann, to incorporate the three-dimensional computer model is identical in each of the image-overlaid models for the previous units used during said training and in said new image-overlaid model (as taught by Park), in order to provide modeling data with minimal consumption of resources and/or risk of damage to equipment (Park; [Col. 5, lines 1-26]).
Allowable Subject Matter
Claims 1, 5-14, and 18-25 are allowed.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
(regarding image modeling and/or detection of aircraft)
Scott et al. (US PGPUB No. 20230362494 A1);
Engelbart et al. (US Patent No. 9645095 B2);
Kelsey et al. (US PGPUB No. 20200211286 A1);
Li et al. (US PGPUB No. 20240420445 A1);
Lowry et al. (US PGPUB No. 20210304390 A1); and
Thompson et al. (US Patent No. 9519844 B1).
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. Refer to PTO-892, Notice of Reference Cited for a listing of analogous art.
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Charles Lloyd Beard whose telephone number is (571)272-5735. The examiner can normally be reached Monday - Friday, 8:00 AM - 5: 00 PM, alternate Fridays EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Tammy Goddard can be reached at (571) 272-7773. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
CHARLES LLOYD. BEARD
Primary Examiner
Art Unit 2611
/CHARLES L BEARD/ Primary Examiner, Art Unit 2611
Prosecution Insights