DETAILED ACTION
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1-20 have been presented for examination based on the application filed on 9/7/2023.
Claim(s) 1-9, 11-18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL by T. Kesavadas et al ("Virtual tools with attributes for robotic based intermediate manufacturing processes," Proceedings of IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA, 1996, pp. 1845-1850 vol.2), in view of US 20210286339 A1 by LINK; Bruce et al..
Claim(s) 14-19 are rejected under 35 U.S.C. 103 as being unpatentable over NPL by T. Kesavadas et al ("Virtual tools with attributes for robotic based intermediate manufacturing processes," Proceedings of IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA, 1996, pp. 1845-1850 vol.2), in view of US 20210286339 A1 by LINK; Bruce et al., further in view of US 20140088746 A1 by Maloney; Max A. et al.
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL by T. Kesavadas et al ("Virtual tools with attributes for robotic based intermediate manufacturing processes," Proceedings of IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA, 1996, pp. 1845-1850 vol.2), in view of US 20210286339 A1 by LINK; Bruce et al., further in view of US 20020133264 A1 by Maiteh, Bilal Y. et al.
This action is made Non-Final.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-9, 11-18 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL by T. Kesavadas et al ("Virtual tools with attributes for robotic based intermediate manufacturing processes," Proceedings of IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA, 1996, pp. 1845-1850 vol.2), in view of US 20210286339 A1 by LINK; Bruce et al..
Regarding Claims 1 & 20
Kesavadas teaches
(Claim 1) A method of performing a finishing operation on a surface of a component (Kesavadas: Pg. 1848 Col.1 ¶2 where the finishing operation is the griding operation of a die as a component Pg. 1848 Col.1 ¶ "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station...."), the method comprising:
(Claim 20) A non-transitory computer readable medium comprising instructions that, when executed by a processing device (Kesavadas: a computer modeling implied in Fig.1, 3; Link: [0158]-[0162] Fig.13) , cause the processing device to perform operations comprising:
acquiring, by a sensing system1, an image of the surface of the component (Kesavadas: Pg.1848 Col.2 ¶1 & 3);
retrieving a computer-aided design (CAD) model corresponding to the surface of the component (Kesavadas: Pg.1848 Col.2 ¶2 "... The workpiece may also be represented mathematically and superimposed with the live object after probing the surface for more accurate correlation. The mathematical description considers the workpiece as a graphic object with its surface defined in a parameterized form as a NURB surface. At..." - mathematical representation using NURBS surface is mapped to CAD model, however Kesavadas also teaches prior art using CAD model specifically also Pg.1846 §3.3);
comparing the image of the surface of the component with the CAD model (Kesavadas: Pg.1847 §4 "... Along with the advantages of carrying dynamics,
control law and other attributes with each virtual tool another aspect which makes the point-and-direct VEPADAnterSpec cell more powerful than a traditional
virtual reality that is interactive but purely graphical, is the ability of the cell to integrate virtual tools in live scenes such that comparison with true reality and the making of corresponding modifications within the scene are now possible....");
identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Kesavadas: Pg. 1849 Col.1 ¶1-2 - ¶1 "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station. He or she can also manipulate the
image and paste the image as a texture on the graphic object in the virtual world. The texture which is a bitwise intensity map of the die surface, unlike the
conventional graphic system can have higher representation than ordinary pixel values. The texture map serves as a method to visualize tht: actual
orientation of the texture...." ¶2"... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or deburring2 [this is based on threshold based user has determined]. One main advantage in such a cell is that the operator can now select smaller segments of the visual cues to give a directive to “grind only from there to there, but avoid area from there to there....") ; and
displaying at least one visual overlay at a location corresponding to the at least one target area on a real-world content or a virtual content (Kesavadas: Fig.3 and §4 "... In the second, a texture mapping of a selected region of video is pasted onto a corresponding geometric object or workpiece for texture analysis and to provide historical detail of the real item in the virtual world....").
Kesavadas does not explicitly teach identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Bolded to show explicitly recited threshold).
Link teaches identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Link: Fig.2 & [0084] , Fig.18A element 1816 & [0179]-[0182], Fig.19 & [0199], Fig.22, [0027]-[0234]; Fig.23
PNG
media_image1.png
448
453
media_image1.png
Greyscale
).
It would have been obvious to one (e.g. a designer) of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Link (2021) to Kesavadas (1996) to complement teachings of surface inspection by comparison of image captured with CAD model with more details. The motivation to combine would have been that Link (2021) to Kesavadas (1996) are analogous art to the instant claim in the same field of surface inspection by comparison of image captured with CAD model to determine state of the die (Link: Fig.2, [0003]-[0005] [0227]-[0234]; Kesavadas: Abstract).
Regarding Claim 2
Kesavadas teaches The method according to Claim 1, wherein the virtual content is selected from a group consisting of a monitor, a projector, an augmented reality (AR) device (Kesavadas: Pg.1848 Col.1¶1-Col.2¶1 use of AR implied, use of overlaid display §4 "... In the second, a texture mapping of a selected region of video is pasted onto a corresponding geometric object or workpiece for texture analysis and to provide historical detail of the real item in the virtual world...."; display in §6; §5 Stereo glasses; §4 AR) .
Regarding Claim 3
Kesavadas teaches The method according to Claim 1, wherein the real-world content includes the surface of the component (Kesavadas: §6 Pg. 1849 Col.1 "... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or
deburring. ..."; §7 actual griding based on virtual tool) .
Regarding Claim 4
Kesavadas teaches The method according to Claim 1, wherein the visual overlay has a (Kesavadas: Pg. 1849 Col.1 ¶1-2 - ¶1 "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station. He or she can also manipulate the
image and paste the image as a texture on the graphic object in the virtual world. The texture which is a bitwise intensity map of the die surface, unlike the
conventional graphic system can have higher representation than ordinary pixel values. The texture map serves as a method to visualize tht: actual
orientation of the texture...." ¶2"... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or deburring [as finishing operations]. One main advantage in such a cell is that the operator can now select smaller segments of the visual cues to give a directive to “grind only from there to there, but avoid area from there to there...."). Link teaches showing the mismatch in color (Link: [0231]-[0233]"... [0231] Compare surface error with inspection threshold [0232] Compute planar surface orientation angles, and compare with inspection tolerance [0233] Illustrate error with “heat” color and ‘fuzz’ line segments to CAD rendering...")
Regarding Claim 5
Kesavadas teaches The method according to Claim 1, wherein the visual overlay has a color a form representing a finishing tool for the finishing operation (Kesavadas: Pg. 1849 Col.1 ¶1-2 - ¶1 "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station. He or she can also manipulate the image and paste the image as a texture on the graphic object in the virtual world. The texture which is a bitwise intensity map of the die surface, unlike the conventional graphic system can have higher representation than ordinary pixel values. The texture map serves as a method to visualize tht: actual orientation of the texture...." ¶2"... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or deburring [finishing tool] . One main advantage in such a cell is that the operator can now select smaller segments of the visual cues to give a directive to “grind only from there to there, but avoid area from there to there...."). Link teaches showing the mismatch in color (Link: [0231]-[0233]"... [0231] Compare surface error with inspection threshold [0232] Compute planar surface orientation angles, and compare with inspection tolerance [0233] Illustrate error with “heat” color and ‘fuzz’ line segments to CAD rendering...")
Regarding Claim 6
Kesavadas teaches The method according to Claim 1, wherein the at least one target area includes a plurality of target areas (Kesavadas: Pg. 1849 Col.1 ¶1-2 - ¶1 "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station. He or she can also manipulate the image and paste the image as a texture on the graphic object in the virtual world. The texture which is a bitwise intensity map of the die surface, unlike the conventional graphic system can have higher representation than ordinary pixel values. The texture map serves as a method to visualize tht: actual orientation of the texture...." ¶2"... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or deburring [finishing tool in Fig.1] . One main advantage in such a cell is that the operator can now select smaller segments of the visual cues to give a directive to “grind only from there to there, but avoid area from there to there...."), and the at least one visual overlay includes a plurality of visual overlays in a (Kesavadas: types of finishing operations can be grinding or deburring and associated tools can be different tools as shown in Fig. 1).
Kesavadas does not teach different colors, which Link teaches (Link: [0155] "... To further distinguish between minute variations of the 3-D registered heat maps, visualizations of these variations can be enhanced by using different colors, textures, and or patterns...."; [0251] "... Through subtractive reasoning [comparison between point cloud and CAD] , the interpolated corrected 3-D point cloud and CAD model are paired [overlaid] and a series of D values are calculated for, and associated with, each point in the point cloud.... The visual representation of the point cloud data may be presented as a set of color-coded point cloud data points, a set of color-coded point cloud data points layered atop an image of registered CAD model, or a color-coded polygon mesh created by forming polygon surfaces between the point cloud data points...."). Motivation to combine is similar to as recited in claim 1.
Regarding Claim 7
Kesavadas teaches The method according to Claim 1, wherein the at least one target area includes a plurality of target areas, and the at least one visual overlay includes a plurality of visual overlays, and wherein the plurality of visual overlays are in a plurality of colors corresponding to types of finishing tools to be used for the finishing operation (Kesavadas: Pg. 1849 Col.1 ¶1-2 - ¶1 "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station. He or she can also manipulate the image and paste the image as a texture on the graphic object in the virtual world. The texture which is a bitwise intensity map of the die surface, unlike the conventional graphic system can have higher representation than ordinary pixel values. The texture map serves as a method to visualize tht: actual orientation of the texture...." ¶2"... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or deburring [finishing tool in Fig.1] . One main advantage in such a cell is that the operator can now select smaller segments of the visual cues to give a directive to “grind only from there to there, but avoid area from there to there...."), and wherein the plurality of visual overlays are in a (Kesavadas: types of finishing operations can be grinding or deburring and associated tools can be different tools as shown in Fig. 1).
Kesavadas does not teach different colors, which Link teaches (Link: [0155] "... To further distinguish between minute variations of the 3-D registered heat maps, visualizations of these variations can be enhanced by using different colors, textures, and or patterns...."; [0251] "... Through subtractive reasoning [comparison between point cloud and CAD] , the interpolated corrected 3-D point cloud and CAD model are paired [overlaid] and a series of D values are calculated for, and associated with, each point in the point cloud.... The visual representation of the point cloud data may be presented as a set of color-coded point cloud data points, a set of color-coded point cloud data points layered atop an image of registered CAD model, or a color-coded polygon mesh created by forming polygon surfaces between the point cloud data points...."). Motivation to combine is similar to as recited in claim 1.
Regarding Claim 8
Kesavadas & Link teaches The method according to Claim 1, further comprising registering, by a processing device, the image of the surface of the component and the CAD model (Kesavadas:§4; Link: Fig.2 2020 and 230 at least, Fig.17, 18A) .
Regarding Claim 9
Kesavadas teaches The method according to Claim 1, further comprising outputting data relating to a type of a finishing tool for the finishing operation on the virtual content (Kesavadas: §5 last paragraph, showing what and where and how (griding etc.), §6 "... The virtual tool concept is the basis of a human machine interface that allows an operator to interact with a system at many levels. When the operator selects a virtual tool by “clicking” on an icon of the desired tool in a virtual toolbox window, a representation of the realworld tool is displayed in live video that is laden with attributes associated with that tool. While the virtual tool
is moved at the speed of normal hand motion to depth correlated locations in the live scene, the real tool may be heavy, springy, or kinematically complex and may
require approaching objects with a particular twist with grasping, grinding, cutting or drilling [different (virtual) tools for different finishing operations, see Fig.1] upon arrival....") .
Regarding Claim 11
Kesavadas teaches The method according to Claim 1, wherein the difference is a difference between a target surface roughness and a measured surface roughness (Kesavadas: §5 with bitmap , §7 "...Using the Interspec system, an operator selected a virtual patch that needed to be smoothed by selecting
the icon for a 1 inch tapered grinding tool...."; Link [0058]-[0063] "...The administrator sets up the GD&T information during the pre-processing, and the set-up associated with the object is then saved in the library of objects database 2010. As part of the process, the GUI tools allow the administrator to: [0059] Manually select datum and measurement planes on the CAD model [0060] Set measurement tolerances [0061] Position [0062] Angle of orientation [0063] Roughness ..." [0155] "... To further distinguish between minute variations of the 3-D registered heat maps, visualizations of these variations can be enhanced by using different colors, textures, and or patterns...."; [0251] "... Through subtractive reasoning [comparison between point cloud and CAD] , the interpolated corrected 3-D point cloud and CAD model are paired [overlaid] and a series of D values are calculated for, and associated with, each point in the point cloud.... "... The CAD model with the color overlay will be generated and saved in a report, and a smoothing process may be applied so that the colors look uniform in the final overlay in the heat map. The smoothing of D value parameters can be performed by averaging or other means, in order to obtain a smooth gradation between color-coded segments of the visual representation of the point cloud data. The visual representation of the point cloud data may be presented as a set of color-coded point cloud data points, a set of color-coded point cloud data points layered atop an image of registered CAD model, or a color-coded polygon mesh created by forming polygon surfaces between the point cloud data points....";) .
Regarding Claim 12
Kesavadas & Link teaches The method according to Claim 1, further performing real-time scanning of the surface of the component while performing the finishing operation on the surface of the component (Kesavadas: §3 & §4 detailing scanning & Link teaches: Fig.16 [0175]"... [0175] FIG. 16 is flow diagram illustrating the processing performed to convert the CAD models into a format useable for interfacing with a corrected 3-D point cloud data obtained from the laser module 200 scans of an inspected object. The CAD models can be dgn, dwg, dxf, or stl file formats, or other formats that allow generation of a CAD file. This process may be performed in real time as the objects are being inspected, or can be performed prior to initiating a quality inspection on objects....") .
Regarding Claim 13
Kesavadas teaches The method according to Claim 1, wherein the component is a die (Kesavadas: Abstract & Introduction "... A remote operator, using instrumented hand gestures, is able to graphically select a virtual tool for a particular
task and place it over the real die surface to enscribe regions where work is needed...") .
Regarding Claim 14
Kesavadas teaches A method of performing a finishing operation on a surface of a component (Kesavadas: Pg. 1848 Col.1 ¶2 where the finishing operation is the griding operation of a die as a component Pg. 1848 Col.1 ¶ "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station...."), the method comprising:
acquiring, by a sensing system3, an image of the surface of the component (Kesavadas: Pg.1848 Col.2 ¶1 & 3);
retrieving (Kesavadas : Pg.1848 use of NURBS model implies and computer and memory) a computer-aided design (CAD) model corresponding to the surface of the component (Kesavadas: Pg.1848 Col.2 ¶2 "... The workpiece may also be represented mathematically and superimposed with the live object after probing the surface for more accurate correlation. The mathematical description considers the workpiece as a graphic object with its surface defined in a parameterized form as a NURB surface. At..." - mathematical representation using NURBS surface is mapped to CAD model, however Kesavadas also teaches prior art using CAD model specifically also Pg.1846 §3.3);
comparing the image of the surface of the component with the CAD model (Kesavadas: Pg.1847 §4 "... Along with the advantages of carrying dynamics,
control law and other attributes with each virtual tool another aspect which makes the point-and-direct VEPADAnterSpec cell more powerful than a traditional
virtual reality that is interactive but purely graphical, is the ability of the cell to integrate virtual tools in live scenes such that comparison with true reality and the making of corresponding modifications within the scene are now possible....");
identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Kesavadas: Pg. 1849 Col.1 ¶1-2 - ¶1 "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station. He or she can also manipulate the
image and paste the image as a texture on the graphic object in the virtual world. The texture which is a bitwise intensity map of the die surface, unlike the
conventional graphic system can have higher representation than ordinary pixel values. The texture map serves as a method to visualize tht: actual
orientation of the texture...." ¶2"... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or deburring4 [this is based on threshold based user has determined]. One main advantage in such a cell is that the operator can now select smaller segments of the visual cues to give a directive to “grind only from there to there, but avoid area from there to there....") ;
displaying at least one visual overlay at a location corresponding to the at least one target area on a real-world content or a virtual content (Kesavadas: Fig.3 and §4 "... In the second, a texture mapping of a selected region of video is pasted onto a corresponding geometric object or workpiece for texture analysis and to provide historical detail of the real item in the virtual world...."); and
outputting data relating to at least one of (Kesavadas: §5-6) .
Kesavadas does not explicitly teach identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Bolded to show explicitly recited threshold).
Link teaches identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Link: Fig.2 & [0084] [0091] , Fig.18A element 1816 & [0179]-[0182], Fig.19 & [0199], Fig.22, [0027]-[0234]; Fig.23
PNG
media_image1.png
448
453
media_image1.png
Greyscale
).
Link teaches CAD model is the memory (Link: [0161] "... For example, the CAD model database 425 may be updated by executing a process to upload CAD models stored on a local memory, or to upload CAD models over the network communication interface...." ).
It would have been obvious to one (e.g. a designer) of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Link (2021) to Kesavadas (1996) to complement teachings of surface inspection by comparison of image captured with CAD model with more details. The motivation to combine would have been that Link (2021) to Kesavadas (1996) are analogous art to the instant claim in the same field of surface inspection by comparison of image captured with CAD model to determine state of the die (Link: Fig.2, [0003]-[0005] [0227]-[0234]; Kesavadas: Abstract).
Regarding Claim 15
Kesavadas teaches The method according to Claim 14, wherein the virtual content is selected from a group consisting of a projector, a headset, glasses, contact lenses, a monitor, and an augmented reality (AR) display (Kesavadas:§4-5 discussing virtual reality, augmented reality (AR) and §5 0 stereo glasses to visualize the overlay) .
Regarding Claim 16
Kesavadas teaches The method according to Claim 14, wherein the real-world content includes the surface of the component (Kesavadas: §6 Pg. 1849 Col.1 "... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or
deburring. ..."; §7 actual griding based on virtual tool) .
Regarding Claim 17
Kesavadas teaches The method according to Claim 14, wherein the data is displayed on the virtual content or output via a voice output (Kesavadas: :§4-5 discussing virtual reality, augmented reality (AR) and §5 0 stereo glasses to visualize the overlay) .
Regarding Claim 18
18. The method according to Claim 14, further comprising performing real-time scanning to obtain real-time information relating to measured surface roughness (Kesavadas: §3-§4 scanning, §5 with bitmap , §7 "...Using the Interspec system, an operator selected a virtual patch that needed to be smoothed by selecting the icon for a 1 inch tapered grinding tool...."; & Link teaches: Fig.16 [0175]"... [0175] FIG. 16 is flow diagram illustrating the processing performed to convert the CAD models into a format useable for interfacing with a corrected 3-D point cloud data obtained from the laser module 200 scans of an inspected object. The CAD models can be dgn, dwg, dxf, or stl file formats, or other formats that allow generation of a CAD file. This process may be performed in real time as the objects are being inspected, or can be performed prior to initiating a quality inspection on objects...."; Link [0058]-[0063] "...The administrator sets up the GD&T information during the pre-processing, and the set-up associated with the object is then saved in the library of objects database 2010. As part of the process, the GUI tools allow the administrator to: [0059] Manually select datum and measurement planes on the CAD model [0060] Set measurement tolerances [0061] Position [0062] Angle of orientation [0063] Roughness ..." [0155] "... To further distinguish between minute variations of the 3-D registered heat maps, visualizations of these variations can be enhanced by using different colors, textures, and or patterns...."; [0251] "... Through subtractive reasoning [comparison between point cloud and CAD] , the interpolated corrected 3-D point cloud and CAD model are paired [overlaid] and a series of D values are calculated for, and associated with, each point in the point cloud.... "... The CAD model with the color overlay will be generated and saved in a report, and a smoothing process may be applied so that the colors look uniform in the final overlay in the heat map. The smoothing of D value parameters can be performed by averaging or other means, in order to obtain a smooth gradation between color-coded segments of the visual representation of the point cloud data. The visual representation of the point cloud data may be presented as a set of color-coded point cloud data points, a set of color-coded point cloud data points layered atop an image of registered CAD model, or a color-coded polygon mesh created by forming polygon surfaces between the point cloud data points....";).
---- This page is left blank after this line ----
Claim(s) 14-19 are rejected under 35 U.S.C. 103 as being unpatentable over NPL by T. Kesavadas et al ("Virtual tools with attributes for robotic based intermediate manufacturing processes," Proceedings of IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA, 1996, pp. 1845-1850 vol.2), in view of US 20210286339 A1 by LINK; Bruce et al., further in view of US 20140088746 A1 by Maloney; Max A. et al.
Regarding Claim 14 (Alternately)
Kesavadas teaches A method of performing a finishing operation on a surface of a component (Kesavadas: Pg. 1848 Col.1 ¶2 where the finishing operation is the griding operation of a die as a component Pg. 1848 Col.1 ¶ "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station...."), the method comprising:
acquiring, by a sensing system5, an image of the surface of the component (Kesavadas: Pg.1848 Col.2 ¶1 & 3);
retrieving (Kesavadas : Pg.1848 use of NURBS model implies and computer and memory) a computer-aided design (CAD) model corresponding to the surface of the component (Kesavadas: Pg.1848 Col.2 ¶2 "... The workpiece may also be represented mathematically and superimposed with the live object after probing the surface for more accurate correlation. The mathematical description considers the workpiece as a graphic object with its surface defined in a parameterized form as a NURB surface. At..." - mathematical representation using NURBS surface is mapped to CAD model, however Kesavadas also teaches prior art using CAD model specifically also Pg.1846 §3.3);
comparing the image of the surface of the component with the CAD model (Kesavadas: Pg.1847 §4 "... Along with the advantages of carrying dynamics,
control law and other attributes with each virtual tool another aspect which makes the point-and-direct VEPADAnterSpec cell more powerful than a traditional
virtual reality that is interactive but purely graphical, is the ability of the cell to integrate virtual tools in live scenes such that comparison with true reality and the making of corresponding modifications within the scene are now possible....");
identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Kesavadas: Pg. 1849 Col.1 ¶1-2 - ¶1 "... The operator can now manipulate the mounted camera to get a close up of the die surface and see it on the virtual work station. He or she can also manipulate the
image and paste the image as a texture on the graphic object in the virtual world. The texture which is a bitwise intensity map of the die surface, unlike the
conventional graphic system can have higher representation than ordinary pixel values. The texture map serves as a method to visualize tht: actual
orientation of the texture...." ¶2"... The virtual cell now provides the means to interactively pick points and patches on the surface of the texture mapped object which need grinding or deburring6 [this is based on threshold based user has determined]. One main advantage in such a cell is that the operator can now select smaller segments of the visual cues to give a directive to “grind only from there to there, but avoid area from there to there....") ;
displaying at least one visual overlay at a location corresponding to the at least one target area on a real-world content or a virtual content (Kesavadas: Fig.3 and §4 "... In the second, a texture mapping of a selected region of video is pasted onto a corresponding geometric object or workpiece for texture analysis and to provide historical detail of the real item in the virtual world...."); and
outputting data relating to at least one of (Kesavadas: §5-6) .
Kesavadas does not explicitly teach identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Bolded to show explicitly recited threshold).
Link teaches identifying at least one target area on the surface of the component where a difference in a geometry between the image of the surface of the component and the CAD model exceeds a threshold (Link: Fig.2 & [0084] , Fig.18A element 1816 & [0179]-[0182], Fig.19 & [0199], Fig.22, [0027]-[0234]; Fig.23
PNG
media_image1.png
448
453
media_image1.png
Greyscale
).
Link teaches CAD model is the memory (Link: [0161] "... For example, the CAD model database 425 may be updated by executing a process to upload CAD models stored on a local memory, or to upload CAD models over the network communication interface...." ).
Maloney teaches outputting data relating to at least one of an amount of material to be removed from the at least target area (Maloney: Fig.19-20 & [0077]-[0078] showing one of material removal being performed in simulation) .
It would have been obvious to one (e.g. a designer) of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Link (2021) to Kesavadas (1996) to complement teachings of surface inspection by comparison of image captured with CAD model with more details. The motivation to combine would have been that Link (2021) to Kesavadas (1996) are analogous art to the instant claim in the same field of surface inspection by comparison of image captured with CAD model to determine state of the die (Link: Fig.2, [0003]-[0005] [0227]-[0234]; Kesavadas: Abstract).
It would have been obvious to one (e.g. a designer) of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Maloney to Kesavadas (1996) to complement teachings of force required to remove material (Maloney:[0077]-[0078] Figs19-20; Kesavadas: §3.2 "... Contact produces force,hence stiffness of the tool, compliance of the'end effector and the amount of material removed plays an important part in trajectory planning....") . The motivation to combine would have been that Maloney to Kesavadas (1996) are analogous art to the instant claim in the same field of material removal simulation during the machining/polishing/finishing process (Maloney: [0077]-[0078]; Kesavadas: §3.2, Abstract).
Regarding Claim 15-18
Alternately claims 15-18 would be rejected in similar manner as rejected above.
Regarding Claim 19
Link & Kesavadas teach The method according to Claim 14, wherein the visual overlay has a color (Link : color [0227]-[0234] showing mismatch; Kesavadas:§5 as bitmap of roughness ) or Maloney teaches a form indicating an amount of material to be removed by the finishing operation and a selected finishing tool to be used for the finishing operation (Maloney: Fig.19-20 & [0077]-[0078] showing one of material removal being performed in simulation).
It would have been obvious to one (e.g. a designer) of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Link (2021) to Kesavadas (1996) to complement teachings of surface inspection by comparison of image captured with CAD model with more details. The motivation to combine would have been that Link (2021) to Kesavadas (1996) are analogous art to the instant claim in the same field of surface inspection by comparison of image captured with CAD model to determine state of the die (Link: Fig.2, [0003]-[0005] [0227]-[0234]; Kesavadas: Abstract).
It would have been obvious to one (e.g. a designer) of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Maloney to Kesavadas (1996) to complement teachings of force required to remove material (Maloney:[0077]-[0078] Figs19-20; Kesavadas: §3.2 "... Contact produces force,hence stiffness of the tool, compliance of the'end effector and the amount of material removed plays an important part in trajectory planning....") . The motivation to combine would have been that Maloney to Kesavadas (1996) are analogous art to the instant claim in the same field of material removal simulation during the machining/polishing/finishing process (Maloney: [0077]-[0078]; Kesavadas: §3.2, Abstract).
---- This page is left blank after this line ----
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over NPL by T. Kesavadas et al ("Virtual tools with attributes for robotic based intermediate manufacturing processes," Proceedings of IEEE International Conference on Robotics and Automation, Minneapolis, MN, USA, 1996, pp. 1845-1850 vol.2), in view of US 20210286339 A1 by LINK; Bruce et al., further in view of US 20020133264 A1 by Maiteh, Bilal Y. et al.
Regarding Claim 10
Teachings of Kesavadas & Link are shown in parent claim 1.
Kesavadas & Link do not teach this limitation.
Maiteh teaches The method according to Claim 1, further comprising generating a voice output with information relating to at least one of an amount of material to be removed from the at least one target area, a type of a finishing tool to be used for the finishing operation, and a type of the finishing operation (Maiteh: "... [0064] The second output device is the audio output device 20. To achieve a realistic simulation of three-dimensional sound effects, a sound card can be used. In this invention, the audio output device can be used to simulate material removal sound and the collisions between the user's body and the other objects. The sound effects provide additional constraints that more realistically simulate an actual machining process....") .
It would have been obvious to one (e.g. a designer) of ordinary skill in the art before the effective filing date of the claimed invention to apply the teachings of Maiteh to Kesavadas to complement teachings of AR material material removal (Maiteh:[64]; Kesavadas: §3.2 "... Contact produces force,hence stiffness of the tool, compliance of the'end effector and the amount of material removed plays an important part in trajectory planning....") . The motivation to combine would have been that Maiteh to Kesavadas (1996) are analogous art to the instant claim in the same field of material removal simulation during the machining/polishing/finishing process (Maiteh: [0064], [0070]; Kesavadas: §3.2, Abstract).
Relevant Prior Art of Record
US 20230127208 A1 by Hauschulz; Steven John teaches automated tool selection based on the final assignment for the surface polishing. This art may be further used in conjunction with Kesavadas to perform finishing.
PNG
media_image2.png
588
604
media_image2.png
Greyscale
PNG
media_image3.png
550
688
media_image3.png
Greyscale
---- This page is left blank after this line ----
Conclusion
All claims are rejected.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Examiner’s Note: Examiner has cited particular columns and line numbers in the references applied to the claims above for the convenience of the applicant. Although the specified citations are representative of the teachings of the art and are applied to specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant in preparing responses, to fully consider the references in their entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the prior art or disclosed by the Examiner.
In the case of amending the claimed invention, Applicant is respectfully requested to indicate the portion(s) of the specification which dictate(s) the structure relied on for proper interpretation and also to verify and ascertain the metes and bounds of the claimed invention.
---- This page is left blank after this line ----
Communication
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AKASH SAXENA whose telephone number is (571)272-8351. The examiner can normally be reached Mon-Fri, 7AM-3:30PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, RYAN PITARO can be reached on (571) 272-4071. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
AKASH SAXENA
Primary Examiner
Art Unit 2188
/AKASH SAXENA/Primary Examiner, Art Unit 2188 Saturday, September 19, 2026
1 Also see L34 (from IP.com), US PGPUB No. 20210286339 Fig.2 for scanning
2 Additionally L34 (from IP.com), US PGPUB No. 20210286339 Fig.2 & 19-¶[0199], Fig.23 perform surface threshold based determination based on difference between scanned part and the CAD model. This art may be used in future.
3 Also see L34 (from IP.com), US PGPUB No. 20210286339 Fig.2 for scanning
4 Additionally L34 (from IP.com), US PGPUB No. 20210286339 Fig.2 & 19-¶[0199], Fig.23 perform surface threshold based determination based on difference between scanned part and the CAD model. This art may be used in future.
5 Also see L34 (from IP.com), US PGPUB No. 20210286339 Fig.2 for scanning
6 Additionally L34 (from IP.com), US PGPUB No. 20210286339 Fig.2 & 19-¶[0199], Fig.23 perform surface threshold based determination based on difference between scanned part and the CAD model. This art may be used in future.
Prosecution Insights