DETAILED ACTION
This non-final office action is responsive to application 18/107,805 as submitted on 09 February 2023.
Claim status is currently pending and under examination for claims 1-20 of which independent claims are 1, 12 and 18.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 3-6, 12, 14-16 and 18-20 are rejected under 35 U.S.C. 102a(2) as being anticipated by Lin et al. (US 20220222049 A1), hereinafter Lin.
With respect to claim 1, Lin teaches:
a computer-implemented method for creating a neural network, comprising (Lin discloses “A computer-implemented method comprises presenting a visual representation of an artificial neural network, the visual representation comprising graphical elements representing layers of the artificial neural network” [Abstract].):
receiving instructions for rendering a graphical representation of a dataset and neural network (Lin discloses “a data import interface, through which the developers upload local datasets for training. After the developers choose a specific dataset, the visual model editor 204 may load the dataset and display the dataset and/or related information” [0032].
Lin discloses “the visual model editor 204 may load a visual model 202, which may be edited and saved by a developer using the visual model editor 204 … the visual model editor 204 may include a canvas of the deep learning visual model that can be edited by the developers through operations like drag-and-drop. … FIG. 3 illustrates a schematic diagram of a visual representation 300 of the artificial neural network in accordance with some implementations of the present disclosure. The visual representation 300 may be displayed on the canvas of the visual model editor 204 and includes graphical elements representing respective layers of the artificial neural network, for example, graphical user interface elements capable of interacting with the developers” [0021-0022].
Lin discloses received user inputs (‘instructions’) can modify a visual representation (‘graphical representation’), “the computing device 100 may receive via the input device 150 a user input, e.g., a drag-and-drop operation. For example, the user may drag and drop one layer onto a canvas. The visual programming module 122 may modify the presented visual representation based on the user input and further update code associated with the artificial neural network” [0019].),
wherein the graphical representation comprises a plurality of software tools for development of datasets (Lin discloses a visual model editor (‘graphical representation’) is comprised of a data interface, data import interface, and a uniform user interface (‘software tools’) that can be used to import, load, and display datasets to use for training a model (‘development of datasets’), “the interface of the visual model editor 204 may also include a data interface for previewing datasets. For example, there may be provided with some public data, such as public datasets, and also a data import interface, through which the developers upload local datasets for training. After the developers choose a specific dataset, the visual model editor 204 may load the dataset and display the dataset and/or related information. For example, the total sample size of the dataset, the sample size of the training set and the sample size of the test set may be all displayed on the interface. For example, a uniform user interface may be provided for the public data and the local data, such that the users can quickly import their own data. The users may preview the uploaded data, train the model and perform inference from the trained model” [0032].)
and neural network nodes interconnected by a plurality of edges (Lin discloses “There may not be a one-to-one correspondence between the nodes and the layers of the deep learning framework. For example, one node may correspond to multiple layers in the deep learning framework or multiple nodes may correspond to one layer in the deep learning framework” [0034]. Lin further discloses “the computing device 100 may display, on a monitor, a visual representation of an artificial neural network, such as various layers and connections between the layers” [0019].
To connect various neural network layers together, nodes of each layer must be connected to nodes in the subsequent layer, therefore nodes are “interconnected by a plurality of edges”.);
converting the graphical representation into source code (Lin discloses “the computing device 100 may display, on a monitor, a visual representation of an artificial neural network, such as various layers and connections between the layers. The computing device 100 may receive via the input device 150 a user input, e.g., a drag-and-drop operation. For example, the user may drag and drop one layer onto a canvas. The visual programming module 122 may modify the presented visual representation based on the user input and further update code associated with the artificial neural network. The code may be provided to the output device 160 as output 180 for the user. For example, the code may be presented to the user on the display together with the visual representation” [0019].);
training the neural network (Lin discloses “The visual programming tool provides model building, training and inference functions through a simple user interface and allows the users to design the model by dragging and dropping the deep learning layers in a simple and visual way. Thus, the development procedure of deep learning can be greatly simplified. This tool can automatically generate training code to enable the users to train the model in accordance with a visual model. In addition, the tool can automatically synchronize the visual model representation with the generated code. When the users modify the visual model representation or the code, the code and/or the visual model representation are automatically updated” [0044-0045].
Lin discloses “when the developers modify the visual representation of the artificial neural network, the intermediate model generator 208 may immediately convert the visual representation into an intermediate representation independent of deep learning frameworks. The intermediate representation may correspond to the visual representation 300 and indicate the parameters and name for each graphical element” [0033]. Lin further discloses “the intermediate representation may be provided to a target code generator 210. For each supported deep learning framework, the target code generator 210 can generate training and/or inferential programs and code, such as TensorFlow Code 212, PyTorch Code 214 or CNTK Code 216” [0038].);
and deploying the neural network (Lin discloses “a job submitter 220 may receive code 212, 214 or 216 and input data 218 and submit the generated programs to the local host 222, cloud 224 or cloud 226 for training or inference. The cloud 224 and 226 may deploy a deep learning cloud platform (also known as machine learning cloud platform) for executing the code or programs. In addition, the programs also may be submitted to a remote work station for execution” [0041].).
With respect to claim 3, Lin teaches:
the method of claim 1, wherein the neural network comprises a convolutional neural network (Lin discloses “As shown in FIG. 3, the graphical element 302 represents the input layer, the graphical element 304 indicates a two-dimensional convolutional layer, the graphical element 306 denotes a two-dimensional max pooling layer, the graphical element 308 represents flattened layer and the graphical element 310 indicates a fully-connected layer” [0023]. See Figure 3 depicting a visual representation of an artificial neural network comprised of a two-dimensional convolutional layer.
Lin discloses “the visual model editor 204 may provide a predefined set of layers of the artificial neural network. The structured layers may be organized into a plurality of tool box panels, such as input/output, Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), fully connected layer, tensor operation and the like” [0030].).
With respect to claim 4, Lin teaches:
the method of claim 1, wherein neural network includes a convolutional 2D layer (Lin discloses “As shown in FIG. 3, the graphical element 302 represents the input layer, the graphical element 304 indicates a two-dimensional convolutional layer, the graphical element 306 denotes a two-dimensional max pooling layer, the graphical element 308 represents flattened layer and the graphical element 310 indicates a fully-connected layer” [0023]. See Figure 3 depicting a visual representation of an artificial neural network comprised of a two-dimensional convolutional layer.).
With respect to claim 5, Lin teaches:
the method of claim 1, wherein the neural network includes a max pooling layer (Lin discloses “As shown in FIG. 3, the graphical element 302 represents the input layer, the graphical element 304 indicates a two-dimensional convolutional layer, the graphical element 306 denotes a two-dimensional max pooling layer, the graphical element 308 represents flattened layer and the graphical element 310 indicates a fully-connected layer” [0023]. See Figure 3 depicting a visual representation of an artificial neural network comprised of a two-dimensional max pooling layer.).
With respect to claim 6, Lin teaches:
the method of claim 1, further comprising enabling user editing of the graphical representation (Lin discloses “the visual model editor 204 may include a canvas of the deep learning visual model that can be edited by the developers through operations like drag-and-drop. Drag-and-drop is a common operation in the graphical user interface and the users may conveniently edit connections between the graphical elements via the drag-and-drop operation. For example, the users may drag and drop one graphical element to or from an input of another graphical element. FIG. 3 illustrates a schematic diagram of a visual representation 300 of the artificial neural network in accordance with some implementations of the present disclosure. The visual representation 300 may be displayed on the canvas of the visual model editor 204 and includes graphical elements representing respective layers of the artificial neural network, for example, graphical user interface elements capable of interacting with the developers. For example, the developers may click a graphical interface element to modify parameters of each layer like shape (also known as data dimension), kernel size and activation function etc. Here, the parameters denote one or more modifiable attributes of respective layers” [0022].
Lin discloses users can change (‘edit’) an interface displaying a dataset (‘graphical representation’) when loading and previewing datasets, “the interface of the visual model editor 204 may also include a data interface for previewing datasets. For example, there may be provided with some public data, such as public datasets, and also a data import interface, through which the developers upload local datasets for training. After the developers choose a specific dataset, the visual model editor 204 may load the dataset and display the dataset and/or related information. For example, the total sample size of the dataset, the sample size of the training set and the sample size of the test set may be all displayed on the interface. For example, a uniform user interface may be provided for the public data and the local data, such that the users can quickly import their own data. The users may preview the uploaded data, train the model and perform inference from the trained model” [0032].).
With respect to claim 12, the rejection of claim 1 is incorporated. The difference in scope being:
An electronic computation device comprising: a processor (Lin discloses “computing device 100 can include, but not limited to, one or more processors or processing units 110, memory 120, storage device 130, one or more communication units 140, one or more input devices 150 and one or more output devices 160” [0011].);
a memory coupled to the processor, the memory containing instructions, that when executed by the processor, cause the electronic computation device to perform the steps of (Lin discloses “The processing unit 110 can be a physical or virtual processor and can execute various processing based on the programs stored in the memory 120” [0013].).
With respect to claim 14, Lin teaches:
the device of claim 12, wherein the memory contains instructions, that when executed by the processor, cause the electronic computation device to create a convolutional neural network (Lin discloses “As shown in FIG. 3, the graphical element 302 represents the input layer, the graphical element 304 indicates a two-dimensional convolutional layer, the graphical element 306 denotes a two-dimensional max pooling layer, the graphical element 308 represents flattened layer and the graphical element 310 indicates a fully-connected layer” [0023].
Lin discloses “the visual model editor 204 may provide a predefined set of layers of the artificial neural network. The structured layers may be organized into a plurality of tool box panels, such as input/output, Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), fully connected layer, tensor operation and the like” [0030].
Lin further discloses “the visual model editor 204 may include a canvas of the deep learning visual model that can be edited by the developers through operations like drag-and-drop. Drag-and-drop is a common operation in the graphical user interface and the users may conveniently edit connections between the graphical elements via the drag-and-drop operation. For example, the users may drag and drop one graphical element to or from an input of another graphical element. FIG. 3 illustrates a schematic diagram of a visual representation 300 of the artificial neural network” [0022]. See Figure 3 depicting a visual representation of an artificial neural network comprised of a two-dimensional convolutional layer that can be edited by users.).
With respect to claim 15, Lin teaches:
the device of claim 12, wherein the memory contains instructions, that when executed by the processor, cause the electronic computation device to create a convolutional 2D layer (Lin discloses “As shown in FIG. 3, the graphical element 302 represents the input layer, the graphical element 304 indicates a two-dimensional convolutional layer, the graphical element 306 denotes a two-dimensional max pooling layer, the graphical element 308 represents flattened layer and the graphical element 310 indicates a fully-connected layer” [0023].
Lin discloses “the visual representation 300 may be displayed on the canvas of the visual model editor 204 and includes graphical elements representing respective layers of the artificial neural network, for example, graphical user interface elements capable of interacting with the developers. For example, the developers may click a graphical interface element to modify parameters of each layer like shape (also known as data dimension), kernel size and activation function etc” [0022]. See Figure 3 depicting a visual representation of an artificial neural network comprised of a two-dimensional convolutional layer that can be edited by users.).
With respect to claim 16, Lin teaches:
the device of claim 12, wherein the memory contains instructions, that when executed by the processor, cause the electronic computation device to create a max pooling layer (Lin discloses “As shown in FIG. 3, the graphical element 302 represents the input layer, the graphical element 304 indicates a two-dimensional convolutional layer, the graphical element 306 denotes a two-dimensional max pooling layer, the graphical element 308 represents flattened layer and the graphical element 310 indicates a fully-connected layer” [0023].
See [0022] describing how layers of a visual representation of an artificial neural network can be edited by developers. See Figure 3 depicting a visual representation of an artificial neural network comprised of a two-dimensional max pooling layer.).
With respect to claim 18, the rejection of claim 1 is incorporated. The difference in scope being:
A computer program product for an electronic computation device comprising a computer readable storage medium having program instructions embodied therewith (Lin discloses “a computer program product tangibly stored in a non-transitory computer storage medium and including computer-executable instructions, wherein the computer-executable instructions, when executed by a device, cause the device to perform the method of the first aspect” [0071].),
the program instructions executable by a processor to cause the electronic computation device to (Lin discloses “a computer program product tangibly stored in a non-transitory computer storage medium and including computer-executable instructions, wherein the computer-executable instructions, when executed by a device, cause the device to perform the method of the first aspect” [0071]. A device executing computer-executable instructions implies a processor.).
With respect to claim 19, the claim recites similar limitations corresponding to claim 15, therefore the same rationale of rejection is applicable.
With respect to claim 20, the claim recites similar limitations corresponding to claim 16, therefore the same rationale of rejection is applicable.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Chen (US 20210056433 A1).
With respect to claim 2, Lin teaches the method of claim 1, however Lin does not teach source code comprising Python, which is taught by Chen:
wherein the source code comprises python (Chen discloses converting a sequence of instruction graphic tags (graphical representation) of neural network layers into a neural network program (‘source code’) , “Taking the commonly used object-oriented programming language Python as an example, the user may select sequentially the input layer instruction graphic tag, the flatten layer instruction graphic tag, the fully connected layer instruction graphic tag, the fully connected layer instruction graphic tag, and the output layer instruction graphic tag, and input the desired parameters in the fields for parameter input or selection in the corresponding functional content graphs, so as to obtain the combination sequence of instruction graphic tags shown in FIG. 3A. Then, the user may request the combination sequence of instruction graphic tags in FIG. 3A to be converted into the neural network program; the control unit 162 in turn will combine the corresponding program sets (the input layer program set, the flatten layer program set, the fully connected layer program set, the fully connected layer program set, and the output layer program set, sequentially) according to the contents in the combination sequence of instruction graphic tags to obtain the neural network program shown in FIG. 3B” [0040]. See Figure 3A depicting a sequence of instruction graphic tags to be converted into a neural network program. See Figure 3B depicting the neural network program (‘source code’) of the converted sequence of instruction graphic tags.
Chen further discloses “combine program sets corresponding to these instruction graphic tags in an order identical to that of the contents in the existing combination sequence of instruction graphic tags to generate a neural network program (for example, but not limited to, a neural network program written in the programming language of Python” [0039].).
Chen teaches converting a sequence of instruction graphic tags (graphical representation) into a Python neural network program (‘source code’) is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the visual representation of Lin with the technique disclosed by Chen to save a graphical representation of a neural network as a Python program. By saving a graphical representation of a neural network as a Python program, the program can remain highly interpretable, allowing users to understand the logic and reasoning behind their neural network model, thus leading to easier debugging and model optimization.
With respect to claim 13, Lin teaches the device of claim 12, however Lin does not teach converting a graphical representation into Python code, which is taught by Chen:
wherein the memory contains instructions, that when executed by the processor, cause the electronic computation device to convert the graphical representation to python code (Chen discloses converting a sequence of instruction graphic tags (graphical representation) of neural network layers into a neural network program (‘source code’) , “Taking the commonly used object-oriented programming language Python as an example, the user may select sequentially the input layer instruction graphic tag, the flatten layer instruction graphic tag, the fully connected layer instruction graphic tag, the fully connected layer instruction graphic tag, and the output layer instruction graphic tag, and input the desired parameters in the fields for parameter input or selection in the corresponding functional content graphs, so as to obtain the combination sequence of instruction graphic tags shown in FIG. 3A. Then, the user may request the combination sequence of instruction graphic tags in FIG. 3A to be converted into the neural network program; the control unit 162 in turn will combine the corresponding program sets (the input layer program set, the flatten layer program set, the fully connected layer program set, the fully connected layer program set, and the output layer program set, sequentially) according to the contents in the combination sequence of instruction graphic tags to obtain the neural network program shown in FIG. 3B” [0040]. See Figure 3A depicting a sequence of instruction graphic tags to be converted into a neural network program. See Figure 3B depicting the neural network program (‘source code’) of the converted sequence of instruction graphic tags.
Chen further discloses “combine program sets corresponding to these instruction graphic tags in an order identical to that of the contents in the existing combination sequence of instruction graphic tags to generate a neural network program (for example, but not limited to, a neural network program written in the programming language of Python” [0039].).
Chen teaches converting a sequence of instruction graphic tags (graphical representation) into a Python neural network program (‘Python code’) is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the visual representation of Lin with the technique disclosed by Chen to save a graphical representation of a neural network as a Python program. By saving a graphical representation of a neural network as a Python program, the program can remain highly interpretable, allowing users to understand the logic and reasoning behind their neural network model, thus leading to easier debugging and model optimization.
Claims 7 and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Mahajan et al. (US 20170132817 A1), hereinafter Mahajan.
With respect to claim 7, Lin teaches the method of claim 6, however Lin does not teach saving a graphical representation into a JSON format, which is taught by Mahajan:
further comprising saving the graphical representation in a JSON format (Mahajan discloses an input PMML file can store a user-modified graphical representation of a neural network, “the system 102 may receive an input file from the user, wherein the input file corresponds to the neural network model to be visualized over a Graphical User Interface (GUI). The input file may be a Predictive Model Markup Language (PMML) file corresponding to the neural network model. The system 102 may further identify the standard template, from a set of standard templates, corresponding to the neural network model. Further, the system 102 may visualize a graphical representation of the neural network model and set of elements corresponding to the neural network. The system 102 may enable the user to modify the graphical representation of the neural network model over the GUI. In one aspect, the system 102 may enable the user to modify relationship between input neurons and output neurons, data flow between the neural nodes of the neural network models, features of the neural network models and others. In one example, the system 102 may edit various neural nodes of the neural network model, the input neural nodes, the output neural nodes, the edges connecting the input neural nodes to the output neural nodes, of the neural network model. Further, the system 102 may update the input file based on the modification of one or more elements from the set of elements of the graphical representation of the neural network model” [0045].
Mahajan discloses “the visualization module 214 may translate the PMML file into JavaScript Object Notation (JSON) format. In one aspect, the visualization module 214 may translate entire information captured in the PMML file into JSON format” [0033].).
Mahajan teaches translating (‘saving’) a user-edited graphical representation of a neural network into JSON format is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the visual representation of Lin with the technique disclosed by Mahajan to save a visual representation of a neural network as a JSON. By saving a visual representation of a neural network as a JSON, a visual representation can be easily exchanged through web applications since JSON is human-readable and lightweight, thus leading to efficient data communication.
With respect to claim 17, the claim recites similar limitations corresponding to claim 7, therefore the same rationale of rejection is applicable.
Claims 8-11 are rejected under 35 U.S.C. 103 as being unpatentable over Lin in view of Burkhardt et al. (US 20210142467 A1), hereinafter Burkhardt.
With respect to claim 8, Lin teaches the method of claim 1, however Lin does not teach training a neural network using supervised learning, which is taught by Burkhardt:
wherein training the neural network comprises training the neural network using supervised learning (Burkhardt discloses “the processing system is configured or otherwise programmed to train a model based on the plurality of training images and corresponding labels associated with each of the plurality of training images” [0009].
Burkhardt discloses “the user may label or classify detected defects as non-defective. Such data may then be fed back into the model that is being used by the AI system 104 (e.g., thus re-training the model)” [0042]. Burkhardt further discloses “The AI system 104 includes a data generator module 202, a model builder module 204, a model training module 206, and a trained (deep) neural network 208” [0043].).
Burkhardt teaches training a neural network with labeled training data (‘supervised learning’) is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the visual representation of Lin with the technique disclosed by Burkhardt to train a neural network using supervised learning. By using supervised learning, neural networks can learn to make predictions by mapping data with its correct label, thus learning underlying patterns and leading to a more accurate model.
With respect to claim 9, the combination of Lin in view of Burkhardt teaches:
The method of claim 8, wherein the supervised learning is based on a plurality of images of a manufacturing process (Burkhardt discloses “Images that are generated without defects may be labeled as non-defective. The generation of images with defects may be accomplished by using a higher range for the various parameters that are provided by a user. For example, by increasing the variance by 50% (or some other factor or absolute value) of the parameters. Thus, for example, when a user defines a 2 pixel variance in the X direction, defective welds may be generated when the seam is greater than (or equal to in certain example embodiments) 3 pixels of variance. … Thus, a dataset of training images that includes images of both non-defective and defective welds may be generated” [0054-0055].
Burkhardt discloses “the generation of training images is repeated until a plurality of training images have been generated. This may be thousands or even millions of different images. The plurality of training images may be varied due to application of the randomness factor that is used in the creation of each one of the plurality of training images. Thus, the plurality of training images may be diverse and robust in nature and include many different examples of defective welds between the plates, including examples that would normally be quite rare” [0010].
Burkhardt further discloses “base images 300 are used as inputs to construct thousands or millions of welding images. In certain example embodiments, base images 300 include two base images that are provided as inputs. These are 1) an image of metallic plate; and 2) an image of weld seam. … Multiple such images may be supplied and the generation of synthetic welding images may be based on a combination of such base images … The images may be close to actual weld plates and weld seams being used in the target manufacturing process. Thus, for example, different manufacturing processes that use different types of plates and/or different welds may be used to generate different input dataset” [0057].).
Burkhardt teaches training a neural network with labeled welding images for a target manufacturing process is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the visual representation of Lin with the neural network disclosed by Burkhardt to use a neural network to predict defects in a manufacturing process. By using a neural network to predict defects in a manufacturing process, defect detection can be accurately and efficiently automated, leading to improved product quality and reliability.
With respect to claim 10, the combination of Lin in view of Burkhardt teaches:
The method of claim 9, wherein the plurality of images includes success workpieces and failure workpieces (Burkhardt discloses images of non-defective welds (‘success workpieces’) and defective welds (‘failure workpieces’) can be generated and labeled, “Images that are generated without defects may be labeled as non-defective. The generation of images with defects may be accomplished by using a higher range for the various parameters that are provided by a user. … Thus, a dataset of training images that includes images of both non-defective and defective welds may be generated” [0054-0055].).
Burkhardt teaches training a neural network with labeled images of non-defective (successful) and defective (failed) welds (‘workpieces’) is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the visual representation of Lin with the labeled images disclosed by Burkhardt to train a neural network with images of non-defective and defective workpieces. By training a neural network with images of non-defective and defective workpieces, a neural network can learn underlying patterns and features to distinguish a non-defective workpiece from a defective workpiece, leading to higher classification accuracy.
With respect to claim 11, the combination of Lin in view of Burkhardt teaches:
The method of claim 10, further comprising: acquiring images of new workpieces (Burkhardt discloses “Camera 112 is configured to capture images 118 of a welding process being performed on the assembly line of the manufacturing process. Camera 112 may be, for example an IoT enabled camera sensor that continuously (e.g., every second, or many times per second) captures images of welds that are being performed (e.g., in progress) and/or have been performed (e.g., are completed). … Images captured by the camera may then be transmitted, via transmitter 114, to AI system 104” [0037].);
and classifying the images based on the neural network (Burkhardt discloses “Deep neural network 208 is the output from the model training module 206 and takes input 210 that is (e.g., continuously) supplied to it and provides outputs 212 based on the model. In other words, the deep neural network 208 may make predictions (whether there is a defect in a weld) based on the incoming data (images of welds). As discussed herein, an example of input 210 are images of a welding process and output is whether there is a detected defect within such images (and potentially the location of the defect within the image)” [0048].
Burkhardt further discloses “in the production environment, real images 118 are then subjected to the model at 314 to provide predictions on the quality of the weld within the provided images. The model is used to scan the provided images using, for example, a sliding window that looks for welding defects. On detection of defects, coordinates of the defect location on welded plates are marked or otherwise stored as defective. An example of this is shown in connection with FIG. 12 where the indicated area with in the analyzed image is blown up to show where the defect is located. In certain examples, this area may be stored using coordinates or the like” [0068].).
Burkhardt teaches capturing real images of welds and predicting defects by using a neural network is a known method in the art. Before the effective filing date of the claimed invention, it would have been obvious to a person of ordinary skill in the art to combine the visual representation of Lin with the technique disclosed by Burkhardt to use a neural network to predict defects in a manufacturing process. By using a neural network to predict defects in a manufacturing process, defect detection can be accurately and efficiently automated, leading to improved product quality and reliability.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Nookula et al. (US 11853401 B1) teaches building a neural network using building blocks representing convolutional layers in a graphical user interface.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PEDRO J MORALES whose telephone number is (571)272-6106. The examiner can normally be reached 8:30 AM - 6:00 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, MIRANDA M HUANG can be reached at (571)270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PEDRO J MORALES/Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124