DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 10 March 2026 have been fully considered but they are not persuasive.
Applicant argues that Meess’s adhesive point markers are not a label, bar code or QR code. Applicant also argues that under BRI the “’feature of the subject item’ consistently means native, manufactured/identification features of the inspected part (e.g. weld studs, nuts, barcodes, QR codes, labels, etc.), not external fiducials added for a tracking system.”
In response, the externally applied vs native features of the inspected “subject item” is considered a false distinction in view of the instant specification and the argument quoted above which defines these features as including QR codes and barcodes while noting that a QR/bar code is not a native feature of, e.g., of a machine part to be welded, but instead a marker that is affixed to the surface of the part. Moreover, the specification and originally filed claims define “feature of the subject item” as including externally affixed fiducials including labels, QR codes, bar codes such that the point marker labels of Meess are clearly within the BRI.
Still further, the claimed “subject item” for which feature presence, location or characteristic is determined is defined by Applicant to include the weld itself. See original and amended claims 1 and 8 which recites “spot weld” as a subject item and presence/absence thereof as a “feature of the subject item”. See also [0032]-[0035], [0039] of the instant specification as published.
Even further, Meess performs workpiece recognition and a workpiece is well within the BRI of “subject item” as established above. See Fig. 26, step 966, [0113] including identifying a range of welding joint types (another “subject item” as claimed) such as butt-type joint, orbital join, T-joint, etc. and the weld paths associated with each joint type. Meess also identifies the weld path associated with the workpiece in [0113] and such weld path is presented on the AR display. Intended weld path display is also considered an augmented reality display that is overlayed onto a live stream of the “subject item”. See also Figs. 22, 24, [0109] in which a virtual weld object 902 is overlaid in the AR display on a completed weld 480D while noting that the completed weld 480D is another “subject item” and Meess presents an AR display (e.g. the virtual weld object 902) onto a live image of the subject item (completed weld 480D).
Still further, features of the completed weld 480D (subject item) include comparison results (determined features of subject item) relating to weld quality (defectiveness) or weld presence (incomplete fusion leaving a gap or a missing weld portion) in which displayed AR attributes 952 include AR overlays for a determined weld profile that is too thin, has too much weld filter or is an incomplete/missing weld as per [0109]-[0111]. Comparing with instant claim 8 further demonstrates the BRI alignment of terms and read of Mees on the claimed invention.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claims 1, 4, 5, 7, 8, 11, 12, 14, 15 and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Meess (US 20180130376 A1) and Antonelli {Dario Antonelli, Sergey Astanin, Enhancing the Quality of Manual Spot Welding through Augmented Reality Assisted Guidance, Procedia CIRP, Volume 33, 2015, Pages 556-561, ISSN 2212-8271, https://doi.org/10.1016/j.procir.2015.06.076.}
Claim 1
In regards to claim 1, Meese discloses an automated inspection system {see abstract, Figs. 1, 13, 14, 17 and cites below including an automated welding inspection system} comprising:
a camera configured to capture an image of a subject item
{Fig. 13, 17 including image/video capture device 470; Fig. 25 tracking system 526, capture image information step 550 [0072]-[0074] captures images of the object to be welded and the weld};
a processor in communication with the camera and programmed to recognize the subject item in the image
{system 400, including processor-based system 410 may include a coupon/workpiece recognition device that recognizes the subject item, [0112]-[0113]. See also spatial tracker 420 that may use Optitrack tracking tools for object tracking which may be incorporated into imaging hardware and software of logical processor based subsystem 410; Fig. 26, [0112]-[0114]
Further, the claimed “subject item” for which feature presence, location or characteristic is determined is defined by Applicant to include the weld itself. See original and amended claims 1 and 8 which recites “spot weld” as a subject item and presence/absence thereof as a “feature of the subject item”. See also [0032]-[0035], [0039] of the instant specification as published. Meess also recognizes weld “subject items” and “workpiece subject items” as per [0113].
Even further, Meess performs workpiece recognition and a workpiece is well within the BRI of “subject item” as established above. See Fig. 26, step 966, [0113] including identifying a range of welding joint types (“subject item” as claimed) such as butt-type joint, orbital join, T-join, etc. and the weld paths associated with each joint type. Meess also identifies the weld path associated with the workpiece in [0113] and such weld path is presented on the AR display. Intended weld path is also considered an augmented reality display that is overlayed onto a live stream of the “subject item”. See also Figs. 22, 24, [0109] in which a virtual weld object 902 is overlaid in the AR display on a completed weld 480D while noting that the completed weld 480D is another “subject item” and Mees presents an AR display (e.g. the virtual weld object 902) onto a live image of the subject item (completed weld 480D)}; and
wherein the processor is configured to determine, from the image, a presence, location, or characteristic of a feature of the subject item.
{various features of the subject item (completed weld 408D) such as color, shape, size, intensity are determined and compared against weld profiles which is a data set regarding a design configuration (e.g. ideal weld object, image database that includes stored models of known welding coupons and workpieces) as per [0069], [0108]-[0113] including identifying the type of the weld path, the length and orientation of the weld path to determine defects (presence, location and characteristics)
In other words, features of the completed weld 480D (subject item) include comparison results (determined features of subject item) relating to weld quality (defectiveness) or weld presence (incomplete fusion leaving a gap or a missing weld portion) in which displayed AR attributes 952 include AR overlays for a determined weld profile that is too thin, has too much weld filter or is an incomplete/missing weld as per [0109]-[0111]. Comparing with instant claim 8 further demonstrates the BRI alignment of terms and read of Meess on the claimed invention.},
wherein the feature of the subject item is one of:
{See above features including features of a completed weld 408D or workpiece including, as per Fig. 26, step 966, [0113] including identifying a range of welding joint types (“subject item” as claimed) such as butt-type joint, orbital join, T-joint, etc. and the weld paths associated with each joint type.},
wherein the processor is further configured to present an augmented reality display as an overlay onto a live image of the subject item
{see [0073]-[0074], [0082]-[0083] including face-mounted display 440a that delivers live full motion video of the welding environment and overlays virtual objects in real time (augmented reality display) per [0103]-[0110], Figs. 18, 22-24.
Mees also identifies the weld path associated with the workpiece in [0113] and such weld path is presented on the AR display. Intended weld path is also considered an augmented reality display that is overlayed onto a live stream of the “subject item”. See also Figs. 22, 24, [0109] in which a virtual weld object 902 is overlaid in the AR display on a completed weld 480D while noting that the completed weld 480D is another “subject item” and Mees presents an AR display (e.g. the virtual weld object 902) onto a live image of the subject item (completed weld 480D).
Still further, features of the completed weld 480D (subject items) include comparison results (determined features of subject item) relating to weld quality or weld presence in which displayed AR attributes 952 include AR overlays for a determined weld profile that is too thin, has too much weld filter or is an incomplete/missing weld as per [0109]-[0111]}.
Although Meess discloses determining a variety of presence, location and characteristic features of various types of subject items including the general category of welds, Meess does not specifically mention wherein the feature of the subject item is one of: a weld stud, a weld stud backing, a weld nut, a weld nut backing, a clinch nut, a spot weld, a bracket, a clip, a split, or a baffle attachment.
Antonelli is analogous art from the same field of automated inspection systems for welding with AR overlays. See abstract, Fig. 4 including camera, welding gun, tracking, and AR engine.
Antonelli also teaches wherein the processor is configured to determine, from the image, a presence, location, or characteristic of a feature of the subject item, and wherein the feature of the subject item is one of: a weld stud, a weld stud backing, a weld nut, a weld nut backing, a clinch nut, a spot weld, a bracket, a clip, a split, or a baffle attachment {see section 3 proposed solution, Table 1 and Figs. 4, 7 and 8 which a workpiece including determining presence/absence/quality (features) of a spot weld (subject item) and generating AR display overlays.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Meese which already determines the presence, absence and location and characteristics of the weld to generate augmented reality display overlays including displaying weld defect presence, location and quality such that this information such that this system is applied to determining presence, location or characteristics of spot-welds for generating AR display overlays as taught by Antonelli because Meess motivates applying his system to a variety of determinations of presence, location and characteristic features of various types of subject items including the general category of welds, because there is a reasonable expectation of success, and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 4
In regards to claim 4, Meess discloses wherein the augmented reality display includes a live video feed showing the subject item {see [0073]-[0074], [0082]-[0083] including face-mounted display 440a that delivers live full motion video of the welding environment and overlays virtual objects in real time (augmented reality display) per [0103]-[0110], Figs. 18, 22-24}.
Claim 5
In regards to claim 5, Meess discloses wherein the augmented reality display includes the overlay presented onto a transparent layer, wherein the subject item is visible to a user through the transparent layer, with the overlay aligned with features of the subject item {see [0073]-[0074], [0082]-[0083] including face-mounted display 440a that delivers live full motion video of the welding environment and overlays aligned virtual objects in real time (augmented reality display) per [0103]-[0110], Figs. 18, 22-24, claim 10, [0063], [0086] wherein the overlay is a transparent layer as claimed.
As noted above, features of the completed weld 480D (subject items) include comparison results (determined features of subject item) relating to weld quality or weld presence in which displayed AR attributes 952 include AR overlays for a determined weld profile that is too thin, has too much weld filter or is an incomplete/missing weld as per [0109]-[0111]}.
Claim 7
In regards to claim 7, Meess discloses
a portable computing device including the camera and a display screen
{Fig. 14 illustrates a portable embodiment including portable stand 520 with support column holding the tracking system including camera, display 430A and/or face mounted display device 440A in the welding helmet 440, [0072]-[0074]}; and
wherein the automated inspection system is configured to present an augmented reality image including one or more overlays onto a live image of the subject item
{see [0073]-[0074], [0082]-[0083] including face-mounted display 440a that delivers live full motion video of the welding environment and overlays aligned virtual objects in real time (augmented reality display) per [0103]-[0110], Figs. 18, 22-24}.
Claim 8
In regards to claim 8, Meess discloses wherein the overlays include: no confirmation icons, one or more confirmation icons indicating a feature being recognized as being present and non-defective, or an error icon indicating a missing or defective feature
{see Figs. 22 and 24 (copied below) including generating and overlaying virtual objects onto the real-world video captures by the helmet-mounted camera 470, [0104]-[0105], [0108]-][0111] including displaying virtual objects 900 that provides feedback to the user to aid in performing the welding operation such as quality of the weld. Fig. 24 illustrates an example that “reports the results of the determination regarding defective features” by changing the color, intensity, etc. of the virtual weld object 952 to indicate that the welding profile is too thin, includes a potential crack, has too much weld filler, the user has stopped short of completing the full weld length (missing weld, defective weld.}
PNG
media_image1.png
294
712
media_image1.png
Greyscale
Independent Claim 11
In regards to claim 11, Meess discloses a method for an automated inspection system, comprising:
tracking a subject item in 3-dimensional space using a feed from a camera viewing the subject item
{Fig. 13, 17 including image/video capture device 470; Fig. 25 tracking system 526, capture image information step 550, and process and correlate tracked data, image data step 530, 540 which tracks a subject item in 3-D space, [0072]-[0074], [0076]-[0082] including spatial tracker 420 that may use Optitrack tracking tools for object tracking which may be incorporated into imaging hardware and software of logical processor based subsystem 410; Fig. 26, [0112]-[0114]};
determining, by the automated inspection system, at least one of a presence, location, or a characteristic of one or more features of the subject item; comparing the at least one of the presence, location, or the characteristic of the one or more features of the subject item with a data set regarding a design configuration to determine if the one or more features are missing or defective
{various features of the subject item (completed weld 408D) such as color, shape, size, intensity are determined and compared against weld profiles which is a data set regarding a design configuration (e.g. ideal weld object, image database that includes stored models of known welding coupons and workpieces) as per [0069], [0108]-[0113] including identifying the type of the weld path, the length and orientation of the weld path to determine defects (presence (missing), location and characteristics),
Further, the claimed “subject item” for which feature presence, location or characteristic is determined is defined by Applicant to include the weld itself. See original and amended claims 1 and 8 which recites “spot weld” as a subject item and presence/absence thereof as a “feature of the subject item”. See also [0032]-[0035], [0039] of the instant specification as published. Mees also recognizes weld “subject items” and “workpiece subject items” as per [0113].
Even further, Mees performs workpiece recognition and a workpiece is well within the BRI of “subject item” as established above. See Fig. 26, step 966, [0113] including identifying a range of welding joint types (“subject item” as claimed) such as butt-type joint, orbital join, T-join, etc. and the weld paths associated with each joint type. Mees also identifies the weld path associated with the workpiece in [0113] and such weld path is presented on the AR display. Intended weld path is also considered an augmented reality display that is overlayed onto a live stream of the “subject item”. See also Figs. 22, 24, [0109] in which a virtual weld object 902 is overlaid in the AR display on a completed weld 480D while noting that the completed weld 480D is another “subject item” and Mees presents an AR display (e.g. the virtual weld object 902) onto a live image of the subject item (completed weld 480D)
Still further, features of the completed weld 480D (subject item) include comparison results (determined features of subject item) relating to weld quality (defectiveness) or weld presence (incomplete fusion leaving a gap or a missing weld portion) in which displayed AR attributes 952 include AR overlays for a determined weld profile that is too thin, has too much weld filter or is an incomplete/missing weld as per [0109]-[0111]. Comparing with instant claim 8 further demonstrates the BRI alignment of terms and read of Mees on the claimed invention.}; and
reporting the results of the determination regarding each of the one or more features being missing or defective
{see Figs. 22 and 24 (copied below) including generating and overlaying virtual objects onto the real-world video captures by the helmet-mounted camera 470, [0104]-[0105], [0108]-][0111] including displaying virtual objects 900 that provides feedback to the user to aid in performing the welding operation such as quality of the weld. Fig. 24 illustrates an example that “reports the results of the determination regarding defective features” by changing the color, intensity, etc. of the virtual weld object 952 to indicate that the welding profile is too thin, includes a potential crack, has too much weld filler, the user has stopped short of completing the full weld length (missing weld, defective weld
PNG
media_image1.png
294
712
media_image1.png
Greyscale
wherein the feature of the subject item is one of:
{See above features including features of a completed weld 408D or workpiece including, as per Fig. 26, step 966, [0113] including identifying a range of welding joint types (“subject item” as claimed) such as butt-type joint, orbital join, T-joint, etc. and the weld paths associated with each joint type.}
Although Meess discloses determining a variety of presence, location and characteristic features of various types of subject items including the general category of welds, Meess does not specifically mention wherein the feature of the subject item is one of: a weld stud, a weld stud backing, a weld nut, a weld nut backing, a clinch nut, a spot weld, a bracket, a clip, a split, or a baffle attachment.
Antonelli is analogous art from the same field of automated inspection systems for welding with AR overlays. See abstract, Fig. 4 including camera, welding gun, tracking, and AR engine.
Antonelli also teaches wherein the processor is configured to determine, from the image, a presence, location, or characteristic of a feature of the subject item, and wherein the feature of the subject item is one of: a weld stud, a weld stud backing, a weld nut, a weld nut backing, a clinch nut, a spot weld, a bracket, a clip, a split, or a baffle attachment {see section 3 proposed solution, Table 1 and Figs. 4, 7 and 8 which a workpiece including determining presence/absence/quality (features) of a spot weld (subject item) and generating AR display overlays.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Meese which already determines the presence, absence and location and characteristics of the weld to generate augmented reality display overlays including displaying weld defect presence, location and quality such that this information such that this system is applied to determining presence, location or characteristics of spot-welds for generating AR display overlays as taught by Antonelli because Meess motivates applying his system to a variety of determinations of presence, location and characteristic features of various types of subject items including the general category of welds, because there is a reasonable expectation of success, and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 12
In regards to claim 12, Meese discloses detecting a part identification of the subject item {see [0068] identifying weld type, [0087], [0113] including spatial tracker 520 and/or logic of system 510 which automatically identifies the objects in the welding environment and the weld path (type, orientation, length, etc.).
Claims 14, 15, 18 and 19
The rejection of system claims 4, 5, 4, and 5 above applies mutatis mutandis to the corresponding limitations of method claims 14, 15, 18 and 19 respectively while noting that the rejection above cites to both device and method disclosures.
Claim 20
In regards to claim 20, Meese discloses wherein the overlay includes: no confirmation icons, one or more confirmation icons indicating a feature being recognized as being present and non-defective, or an error icon indicating a missing or defective feature {see Figs. 22 and 24 (copied below) including generating and overlaying virtual objects (icons) onto the real-world video captures by the helmet-mounted camera 470, [0104]-[0105], [0108]-][0111] including displaying virtual objects 900 that provides feedback to the user to aid in performing the welding operation such as quality of the weld. Fig. 24 illustrates exemplary icons indicating defective features by changing the color, intensity, etc. of the virtual weld object 952 to indicate that the welding profile is too thin, includes a potential crack, has too much weld filler, the user has stopped short of completing the full weld length (missing weld, defective weld)}
Claim 6 and 16-17 are rejected under 35 U.S.C. 103 as being unpatentable over Meess and Antonelli as applied to claims 1 and 11 above, and further in view of Becker (WO 2015/105580).
Claims 6 and 17
In regards to claims 6 and 17, Meese is not relied upon to disclose wherein the processor is configured to generate an inspection report based on determining the presence, location, or characteristic of the feature of the subject item.
Becker is analogous art from the same field of automated inspection. See Fig. 23, [0118] including integrated welding inspection environment with AR displays.
Becker also teaches wherein the processor is configured to generate an inspection report based on determining the presence, location, or characteristic of the feature of the subject item {Fig. 16, [0103]-[0104] including display-menu driven selection of various elements such as welding data reports and generating an inspection report including discontinuity analysis, Fig. 21 [0115] based on the presence, location, or characteristic of the feature of the weld}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Meese which already determines the presence, absence and location and characteristics of the weld to generate augmented reality display overlays including displaying weld defect presence, location and quality such that this information is presented in form of a report wherein the processor is configured to generate an inspection report based on determining the presence, location, or characteristic of the feature of the subject item as taught by Becker because a summary report such as that shown in Fig. 21 of Becker permits further correlation and determination of the source of the weld quality defects thus providing the welding trainee with valuable feedback as motivated by Becker in [0115], because there is a reasonable expectation of success, and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 16
In regards to claims 16, Meese discloses presenting a menu with a list of functions,
{See the face mounted display which has a menu for configuration and operation of the system as per [0074], [0087], [0091]
Becker also teaches presenting a menu with a list of functions, the functions including showing an inspection part list, starting a new inspection, continuing an existing inspection, or reviewing inspection reports
{Fig. 16, [0103]-[0104] including display-menu driven selection of various elements such as welding data reports and generating and showing/reviewing an inspection report including discontinuity analysis, Fig. 21 [0115] based on the presence, location, or characteristic of the feature of the weld}.
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Meese which already employs a menu for configuration and operation of the system such that Becker’s menu functionality is used to present a menu with a list of functions, the functions including showing an inspection part list, starting a new inspection, continuing an existing inspection, or reviewing inspection reports as taught by Becker because a summary report such as that shown in Fig. 21 of Becker permits further correlation and determination of the source of the weld quality defects thus providing the welding trainee with valuable feedback as motivated by Becker in [0115], because there is a reasonable expectation of success, and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claims 9 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Meess and Antonelli as applied to claim 7 above, and further in view of and Nickel (US 20190301845 A1).
Claims 9 and 10
Meese discloses (claim 9) wherein the portable computing device further comprises an
Nickel is analogous art from the same field of automated inspection. See abstract, Figs. 10, 14A, 14B, 15, 17, 18 and their corresponding descriptions.
Nickel is one of many references demonstrating the conventional nature of locating a light source in a portable computing device for automated inspection including the broadly recited light source (illuminator). See light (laser 534) which illuminates the subject item and is an internal illuminator disposed within the portable computing device as illustrated in Figs. 4D-7C (copied above). Furthermore, Nickel’s light is also “an illuminator removably attached to the portable computing device and configured to illuminate the subject item” as illustrated in the cited figures and [0072]-[0086]. See also [0056]-[0060] also teaching the equivalence of integrated as well as detachable components.
PNG
media_image2.png
860
640
media_image2.png
Greyscale
It would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to have modified Meese which already discloses an illuminator to illuminate the subject item such that this illuminator is an internal illuminator disposed within the portable computing device and/or an illuminator removably attached to the portable computing device as taught by Nickel because Nickel teaches that these conventional light locations are equivalents, because there is a reasonable expectation of success, and/or because doing so merely combines prior art elements according to known methods to yield predictable results of illumination.
Conclusion
https://doi.org/10.1016/j.procir.2015.06.076) discloses an AR system for welding defect detection that displays various icons/symbols indicating welding spot quality as per Table 1, fig .4 copied below.
PNG
media_image3.png
386
558
media_image3.png
Greyscale
PNG
media_image4.png
416
524
media_image4.png
Greyscale
Zhou (Jianlong Zhou, Ivan Lee, Bruce Thomas, Roland Menassa, Anthony Farrant, and Andrew Sansome. 2011. Applying spatial augmented reality to facilitate in-situ support for automotive spot welding inspection. In Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry (VRCAI '11). Association for Computing Machinery, New York, NY, USA, 195–200. https://doi.org/10.1145/2087756.2087784) discloses an AR system for spot welding that projects overlays for locating welding locations as per Fig. 6 copied below.
PNG
media_image5.png
294
490
media_image5.png
Greyscale
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Michael R Cammarata whose telephone number is (571)272-0113. The examiner can normally be reached M-Th 7am-5pm EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at 571-272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MICHAEL ROBERT CAMMARATA/Primary Examiner, Art Unit 2667
Prosecution Insights