DETAILED ACTION
This Office action is in response to the amendment filed on September 11, 2026.
Claims 1-20 are pending.
Claims 1, 7, and 14 are currently amended.
The objections to Claims 1, 7, and 14 are withdrawn in view of the Applicant’s amendments to the claims.
The non-statutory obviousness-type double patenting rejections of Claims 1-20 as being unpatentable over Claims 1-13 of U.S. Patent No. 12,079,600 (hereinafter “‘600”) are withdrawn in view of the Applicant’s submission of a terminal disclaimer and approval thereof.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Internet Communications
Without a written authorization for Internet communications by the Applicant in place, the USPTO cannot communicate with the Applicant via email and will not respond via email to any Internet correspondence which contains information subject to the confidentiality requirement as set forth in 35 U.S.C. § 122, such as claimed subject matter in an interview agenda or proposed claim amendments for an Examiner’s Amendment.
Therefore, in the interest of facilitating compact prosecution, the Examiner kindly asks the Applicant to authorize Internet communications with the USPTO by using Form PTO/SB/439 (available at https://www.uspto.gov/patents/apply/forms). The form may be submitted via the USPTO patent electronic filing system (Patent Center) using the document description “Internet Communications Authorized” to facilitate processing. The written authorization for Internet communications must be submitted on a separate paper to be entitled to acceptance in accordance with 37 CFR § 1.4(c). The separate paper will facilitate processing and avoid confusion. The written authorization for Internet communications may not be submitted via an email. See MPEP § 502.03(II).
Claim Interpretation
During patent examination, the pending claims must be “given their broadest reasonable interpretation consistent with the specification.” See MPEP § 2111. Under a broadest reasonable interpretation (BRI), words of the claim must be given their plain meaning, unless such meaning is inconsistent with the specification. The plain meaning of a term means the ordinary and customary meaning given to the term by those of ordinary skill in the art at the relevant time. The ordinary and customary meaning of a term may be evidenced by a variety of sources, including the words of the claims themselves, the specification, the drawings, and the prior art. See MPEP § 2111.01(I).
Applicant is entitled to be their own lexicographer and may rebut the presumption that claim terms are to be given their ordinary and customary meaning by clearly setting forth a definition of the term that is different from its ordinary and customary meaning(s) in the specification at the relevant time. Where an explicit definition is provided by the Applicant for a term, that definition will control interpretation of the term as it is used in the claim. See MPEP § 2111.01(IV)(A). Any such lexicographic definition for a term will be expressly noted by the Examiner in the prior art rejections of the claims.
Claim Mapping
For clarity of the prosecution history record, the Examiner has provided annotations in the prior art rejections of the claims to aid the Applicant in understanding the Examiner’s interpretations of the claimed invention and the prior art, such as emphasizing notable and relevant portions of the prior art citations, using item-to-item matching to the prior art citations, pairing exact claim language to particular language used in the prior art citations, and/or clearly explaining the Examiner’s interpretation as to how a prior art citation maps to the claim language, especially when there is no one-to-one matching of terms. Furthermore, the annotations are provided in the prior art rejections of the claims at the Examiner’s discretion where the Examiner deemed to be appropriate and necessary.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR § 1.55.
Terminal Disclaimer
The terminal disclaimer filed on September 11, 2026 disclaiming the terminal portion of any patent granted on this application which would extend beyond the expiration date of U.S. Patent No. 12,079,600 has been reviewed and is accepted. The terminal disclaimer has been recorded.
Response to Amendment
<<>> • × • <<>>
Claim Rejections - 35 U.S.C. § 103
The following is a quotation of 35 U.S.C. § 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 6-9, 12, 14-16, and 19 are rejected under 35 U.S.C. § 103 as being unpatentable over US 2007/0094168 (hereinafter “Ayala”) in view of US 2020/0349052 (hereinafter “Wehr”) and US 2019/0303509 (hereinafter “Greene”).
EXAMINER’S REMARKS
In order for a reference to be proper for use in an obviousness rejection under 35 U.S.C. § 103, the reference must be analogous art to the claimed invention. In re Bigio, 381 F.3d 1320, 1325, 72 USPQ2d 1209, 1212 (Fed. Cir. 2004). A reference is analogous art to the claimed invention if: (1) the reference is from the same field of endeavor as the claimed invention (even if it addresses a different problem); or (2) the reference is reasonably pertinent to the problem faced by the inventor (even if it is not in the same field of endeavor as the claimed invention). See MPEP § 2141.01(a)(I).
The claimed invention is generally directed to visual programming for deep learning (Abstract). As for the “same field of endeavor” test, Ayala is generally directed to the design and evaluation of artificial neural networks (specification, paragraph [0004]). As for the “reasonably pertinent” test, Wehr is generally directed to analyzing source code using deep learning (specification, paragraph [0001]). And Greene is generally directed to querying application metadata objects (Abstract). Thus, Ayala, Wehr, and Greene are all analogous art to the claimed invention (even if they address different problems or are not in the same field of endeavor as the claimed invention).
As per Claim 1, Ayala discloses:
A computer-implemented method (paragraph [0166], “[…] practice of the disclosed […] method is not limited to any particular programming language.”) comprising:
presenting a visual representation of an artificial neural network (paragraph [0062], “In some cases, the editor interface may provide a panel or pallet for graphically establishing [presenting a visual representation] the structure or topology of the artificial neural network, while other editor interfaces may provide such configuration functionality via a number of input data panels, tables, and other graphical user interface selection items (emphasis added).”), wherein the visual representation of the artificial neural network includes graphical elements representing layers of the artificial neural network (paragraph [0081], “With the identification of each layer, graphical representations of the neurons are depicted in a network topology panel 168 that generally forms a pallet for graphical configuration of the network topology or structure [graphical elements representing layers of the artificial neural network].”);
receiving a drag-and-drop operation on the graphical elements, the drag-and-drop operation configured to modify connections between the graphical elements including automatically connecting a first graphical element to a second graphical element of the graphical elements (paragraph [0081], “A user may then employ the mouse pointer to connect the neurons by clicking on one neuron and dragging the mouse pointer toward another neuron. A form of ‘drag-and-drop’ operation then forms the connection [receiving a drag-and-drop operation on the graphical elements, the drag-and-drop operation configured to modify connections between the graphical elements including automatically connecting a first graphical element to a second graphical element of the graphical elements] (emphasis added).”);
changing, in response to receiving the drag-and-drop operation on the graphical elements (paragraph [0081], “A user may then employ the mouse pointer to connect the neurons by clicking on one neuron and dragging the mouse pointer toward another neuron. A form of ‘drag-and-drop’ operation then forms the connection [in response to receiving the drag-and-drop operation on the graphical elements] (emphasis added).”), an intermediate representation of the artificial neural network (paragraph [0088], “As is known to those skilled in the art, such discriminant planes, or decision lines, provide a representation of the network by a linear or discriminant equation for classification of data points with respect to the line or plane. To that end, the selection of the hyperplanes tab in the panel 174 may provide options or settings directed to a number of discriminants for a layer of the network, as well as for identification of the reference neurons for the hyperplanes equations. The user may then specify the reference neurons for plotting the data input patterns. Once the foregoing settings are established, the data may be plotted in a customizable graph within the panel 174. In this manner, the hyperplanes provide for classification or class separation and facilitate a visualization of the regions created by the network being configured after, for instance, a training iteration [changing […] an intermediate representation of the artificial neural network].”); and
modifying, based on the intermediate representation of the artificial neural network, code of the artificial neural network […] (paragraph [0090], “These settings may be similar or related to the graphical settings made available to the user for customization via the panel 172 of the display interface window 160. The options pull down menu may also provide a mechanism for initiating implementation of the code insight module 116 (FIG. 1) in which case a separate display interface window may be generated to display code related to, for instance, the back propagation training algorithm associated with the feedforward network being configured [modifying, based on the intermediate representation of the artificial neural network, code of the artificial neural network […]].”).
Ayala does not explicitly disclose:
the intermediate representation of the artificial neural network being embodied as a directed graph that is independent of a deep learning framework.
However, Wehr discloses:
the intermediate representation of the artificial neural network being embodied as a directed graph that is independent of a deep learning framework (paragraph [0027], “[…] deep learning (also known as deep structured learning or hierarchical learning) is part of a broader family of machine learning methods based on learning data representations as opposed to task-specific algorithms. Learning can be supervised, semi-supervised or unsupervised (emphasis added).”; paragraph [0039], “An example of an artificial neural network is a recurrent neural network where connections between nodes form a directed graph along a temporal sequence which allows it to exhibit temporal dynamic behavior (emphasis added).”).
As pointed out hereinabove, Ayala and Wehr are both analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Wehr into the teaching of Ayala to include “the intermediate representation of the artificial neural network being embodied as a directed graph that is independent of a deep learning framework.” The modification would be obvious because one of ordinary skill in the art would be motivated to utilize a recurrent neural network where connections between nodes form a directed graph along a temporal sequence which allows it to exhibit temporal dynamic behavior (Wehr, paragraph [0039]).
The combination of Ayala and Wehr discloses “layers specific to the target deep learning framework” (Wehr, paragraph [0027], “[…] deep learning (also known as deep structured learning or hierarchical learning) is part of a broader family of machine learning methods based on learning data representations as opposed to task-specific algorithms. Learning can be supervised, semi-supervised or unsupervised (emphasis added).”), but the combination of Ayala and Wehr does not explicitly disclose:
wherein nodes of the directed graph are converted into code that implements layers specific to the target deep learning framework.
EXAMINER’S REMARKS
Wehr discloses that deep learning is part of a broader family of machine learning methods. Thus, one of ordinary skill in the art would readily comprehend that deep learning focuses on utilizing multilayered neural networks to perform tasks such as classification, regression, and representation learning.
However, Greene discloses:
wherein nodes of the directed graph are converted into code that implements [hierarchical relationships] (Figure 4D; paragraph [0052], “[…] if integration software 120 is written in JAVA language, binder program 314 can be part of a JAVA API to convert the nodes of the directed graph into instances of JAVA objects. […] FIG. 4D illustrates an example of JAVA code 440 converted from directed graph 410 of FIG. 4B (emphasis added).”; paragraph [0053], “As shown in FIG. 4D, the metadata objects that correspond to the vertices of directed graph 410 can be represented as JAVA classes. For example, the classes ‘application_resource,’ ‘contact,’ ‘address,’ and ‘prefer_contact’ correspond to, respectively, the metadata objects ‘application_resource,’ ‘contact,’ ‘address,’ and ‘prefer_contact.’ The JAVA classes also represent the hierarchical relationships among the metadata objects and attributes shown in directed graph 410.”).
As pointed out hereinabove, Greene is an analogous art to the claimed invention. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of Greene into the combined teachings of Ayala and Wehr to include “wherein nodes of the directed graph are converted into code that implements layers specific to the target deep learning framework.” The modification would be obvious because one of ordinary skill in the art would be motivated to convert a directed graph into a format compatible with code of an artificial neural network (Greene, paragraph [0052]).
As per Claim 2, the rejection of Claim 1 is incorporated; and Ayala further discloses:
in response to an editing operation on the code of the artificial neural network for the target deep learning framework, modifying the intermediate representation of the artificial neural network (paragraph [0088], “As is known to those skilled in the art, such discriminant planes, or decision lines, provide a representation of the network by a linear or discriminant equation for classification of data points with respect to the line or plane. To that end, the selection of the hyperplanes tab in the panel 174 may provide options or settings directed to a number of discriminants for a layer of the network, as well as for identification of the reference neurons for the hyperplanes equations. The user may then specify the reference neurons for plotting the data input patterns. Once the foregoing settings are established, the data may be plotted in a customizable graph within the panel 174. In this manner, the hyperplanes provide for classification or class separation and facilitate a visualization of the regions created by the network being configured after, for instance, a training iteration.”); and
adjusting the visual representation of the artificial neural network based on the intermediate representation of the artificial neural network (paragraph [0090], “For instance, in this embodiment, the options available for customization by the user include a number formatting option, a background height option, and a series or set of graphs settings options to specify background colors or styles, as well as graphical item sizes. These settings may be similar or related to the graphical settings made available to the user for customization via the panel 172 of the display interface window 160. The options pull down menu may also provide a mechanism for initiating implementation of the code insight module 116 (FIG. 1) in which case a separate display interface window may be generated to display code related to, for instance, the back propagation training algorithm associated with the feedforward network being configured. Lastly, the options pull down menu may provide a mechanism for saving one or more of the settings specified by the user, such as saving the graph itself set forth in the network topology panel 168.”).
As per Claim 3, the rejection of Claim 1 is incorporated; and Ayala further discloses:
in response to receiving the drag-and-drop operation on the graphical elements (paragraph [0081], “A user may then employ the mouse pointer to connect the neurons by clicking on one neuron and dragging the mouse pointer toward another neuron. A form of ‘drag-and-drop’ operation then forms the connection.”), validating dimensions of data associated with the layers of the artificial neural network (paragraph [0117], “[…] the learning pattern module 108 may support the customization and setup of pattern data sets for cross-validation techniques, as well as perform analysis of the pattern data sets to determine their suitability for use in training and/or testing the neural network.”).
As per Claim 6, the rejection of Claim 1 is incorporated; and Ayala further discloses:
modifying the intermediate representation of the artificial neural network in response to at least one of:
adding, into the visual representation of the artificial neural network, a new graphical element representing a layer of the artificial neural network;
deleting, from the visual representation of the artificial neural network, a graphical element representing a layer of the artificial neural network (paragraph [0096], “The commands made available via the menu bar 202 and the command bar 204 generally enable a user to manage the pattern data set(s) by adding one or more patterns, deleting one or more patterns and saving the pattern data set. In the exemplary case shown in FIG. 9, an icon 206 is included the command bar 204 to delete a number of rows (i.e., patterns) specified in an input data field 207, with the location of the rows to be deleted specified via additional input data fields 208 and 209. Alternatively, or additionally, patterns or rows of the table may be deleted via command key strokes or any other desired interface mechanism.”); and
modifying parameters of a graphical element representing a layer of the artificial neural network.
Claims 7-9 and 12 are device claims corresponding to the computer-implemented method claims hereinabove (Claims 1-3 and 6, respectively). Therefore, Claims 7-9 and 12 are rejected for the same reasons set forth in the rejections of Claims 1-3 and 6, respectively.
Claims 14-16 and 19 are non-transitory machine-readable medium claims corresponding to the computer-implemented method claims hereinabove (Claims 1-3 and 6, respectively). Therefore, Claims 14-16 and 19 are rejected for the same reasons set forth in the rejections of Claims 1-3 and 6, respectively.
Allowable Subject Matter
Claims 4, 5, 10, 11, 13, 17, 18, and 20 are objected to as being dependent upon a rejected base claim under 35 U.S.C. § 103, but would be allowable over the cited prior art if rewritten in independent form including all of the limitations of the base claim and any intervening claims, and overcome any corresponding objections and/or rejections set forth hereinabove.
Reasons for Allowance
The following is an Examiner’s statement of reasons for the indication of allowable subject matter:
As per Claims 4, 10, and 17, the closest cited prior art, the combination of Ayala, Wehr, and Greene fails to teach or suggest, among the other claimed limitations, “in response to receiving a search operation associated with a keyword, presenting graphical elements representing at least one candidate layer corresponding to the keyword; and in response to receiving a selection of graphical elements of the at least one candidate layer, adding the selected graphical elements of the at least one candidate layer to the visual representation of the artificial neural network.” These claimed limitations, in combination with the other claimed limitations, are neither taught nor suggested by the combination of Ayala, Wehr, and Greene.
As per Claims 5, 11, and 18, the closest cited prior art, the combination of Ayala, Wehr, and Greene fails to teach or suggest, among the other claimed limitations, “presenting code stubs for customizing metrics of the artificial neural network; and in response to an editing operation on the code stubs, customizing the metrics of the artificial neural network.” These claimed limitations, in combination with the other claimed limitations, are neither taught nor suggested by the combination of Ayala, Wehr, and Greene.
As per Claims 13 and 20, the closest cited prior art, the combination of Ayala, Wehr, and Greene fails to teach or suggest, among the other claimed limitations, “in response to receiving an instruction for changing the target deep learning framework to a further target deep learning framework, determining code of the artificial neural network for the further target deep learning framework based on the intermediate representation of the artificial neural network.” These claimed limitations, in combination with the other claimed limitations, are neither taught nor suggested by the combination of Ayala, Wehr, and Greene.
Response to Arguments
Applicant’s arguments1 filed on September 11, 2026 with respect to the 35 U.S.C. § 103 rejections of the claims have been fully considered, but they are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the Applicant’s arguments.
Conclusion
Applicant’s amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR § 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR § 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the Examiner should be directed to Qing Chen whose telephone number is 571-270-1071. The Examiner can normally be reached on Monday through Friday from 9:00 AM to 5:00 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, the Applicant is encouraged to use the USPTO Automated Interview Request (AIR) at https://www.uspto.gov/ interviewpractice.
If attempts to reach the Examiner by telephone are unsuccessful, the Examiner’s supervisor, Wei Mui, can be reached at 571-272-3708. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for more information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO customer service representative, call 800-786-9199 (in USA or Canada) or 571-272-1000.
/Qing Chen/
Primary Examiner, Art Unit 2191
1 Applicant’s arguments contain a number of statements reflecting characterizations of the claims and the cited prior art. Regardless of whether any such statement is identified herein, the Examiner declines to automatically subscribe to any statement or characterization made in the Applicant’s arguments.
