Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This Office Action is responsive to Applicants' Amendment filed on 08/07/2025, in which claims 1, 5, 7, 9, 14, 16, 17, and 19 have been amended. Claims 3, 4, 8, 12, 13, and 18 have been cancelled. No new claims have been added.
Claims 1, 2, 5-7, 9-11, 14-17, 19, and 20 are currently pending.
Response to Arguments
With regards to the objections to claims 5, 14, and 19, Applicant has amended the claims to correct the noted minor informalities, and thus the objections are withdrawn.
With regards to the rejection of claim 8 under 35 U.S.C. 112(b) as having indefinite scope, claim 8 has been cancelled and the rejection is moot.
With regards to the rejections of claims 1-20 under 35 U.S.C. 101 as directed towards abstract ideas, Applicant’s arguments that the claims as amended overcome the rejections have been found persuasive. Any recited abstract ideas are incorporated into a practical solution for optimizing a data streaming graph for a deep learning acceleration chip.
With regards to the rejections of claims 1-20 under 35 U.S.C. 103 as unpatentable over Fengbin et al. (Chinese Patent Application Publication No. 106547522), further in view of Nicol (U.S. Patent Application Publication No. 2019/0279038), Applicant’s arguments that the claims as amended overcome the rejections have been considered but are not found persuasive.
Applicant first argues on page 13 of the Remarks that “Applicant notes that the problem the present application seeks to solve is that the computation efficiency is low in the general data streaming data structure design. To solve this problem, the present application obtains a target data streaming computation graph according to parameters in the initial constant expressions and then obtaining a final data streaming computation graph” and “In contrast, the problem of Fengbin is that after an application is deployed on a stream computing platform, if developers do not actively update the operator codes of the stream application, the operator codes of the stream application will basically remain unchanged, and the performance of the stream application cannot be further improved”, and further argues on page 14 of the Remarks that while the instant application obtains operators “by fusing” preceding operators, “Fengbin relates to ‘optimizing operating code (or optimizing the data flow graph of the operator)’…Therefore, the present application and Fengbin take different technical measures to solve different technical problems, and thus are different technical solutions”.
Examiner respectfully disagrees with this assessment. Page 2 of the instant application’s specification states “Embodiments of the present application provide a method and device for adjusting a deep learning network, a server and a storage medium to achieve the effect of improving the
computation efficiency of deep learning networks in the data streaming architecture”. Fengbin states: (Fengbin [0007]) “The embodiment of the present invention provides a method for stream application optimization, which can automatically optimize operator codes, thereby improving stream application performance”. The instant application relates to increasing computation efficiency of data steaming architecture, Fengbin relates to optimizing stream applications, and Examiner considers these to be the same concept expressed in different words. Although Fengbin does not use the precise terminology “fusing” for operators, the broadest reasonable interpretation of fusing includes the migrating of operator logic disclosed by Fengbin. The additional element of the field of the instant application not disclosed by Fengbin is the element of deep learning networks, however this was previously disclosed by Examiner, and Nicol is relied upon to teach this additional element.
Applicant further identifies a first feature “’a granularity of the second operator is larger than a granularity of one of the first operators to enable an adjustment of an amount of computation’”, recited within claim 1 as amended, as not taught by Fengbin. Applicant states on page 14 of the Remarks that “In fact, when migrating the first operation logic or the second operation logic between the upstream operator and the downstream operator, the logical operation complexity is merely used to determine the migrating direction (see paragraphs [0123]-[0125] )” which “cannot adjust an amount of computation”.
Examiner respectfully disagrees that Fengbin does not teach said first feature. Fengbin states: (Fengbin [0109]) “As shown in FIG. 9 , by saving the previous calculation result C, A+=AIN0, C=A*B can be modified to C+==AIN0*B…It can be seen that C can be obtained by calculating only once each time, which reduces the amount of calculation and improves the calculation efficiency”. Migration of operation logic is described by Fengbin as part of (Fengbin [0001]) “a method for stream application optimization, which can automatically optimize operator codes, thereby improving the usability and computing performance of stream applications”. Examiner considers Fengbin sufficiently discloses that optimization of operator codes can adjust an amount of computation by reducing an amount of calculation.
Applicant further identifies a second feature “’acquiring, from among the plurality of second operators, at least two second operators for computing a same target expression; fusing the at least two second operators to obtain a third operator; obtaining a final data streaming computation graph based on a non-fused second operator in the target data streaming computation graph and the third operator’”, recited within claim 1 as amended, as not taught by Fengbin. Applicant states on page 15 of the Remarks that “According to feature (2), fusing at least two second operators for computing a same target expression to obtain a third operator. For example, operator B1 and operator B2 compute the same target expression Y = a x X + b, and operator B1 and operator B2 can be fused to obtain a third operator C1 to compute the target expression Y = a x X +b” and “the present application fuses multiple operators for computing a same target expression to obtain one operator, so as to increase operator granularity to improve computation efficiency…FIG. 12 of Fengbin is totally different from feature (2) of the present application”.
Examiner respectfully disagrees that Fengbin does not teach said second feature. Fengbin Fig. 12 shows acquiring, from among the plurality of second operators, at least two second operators for computing a same target expression; (within the left box, the circled operators compute the same target expression, which is added to F), fusing the at least two second operators to obtain a third operator; (within the right box, the former circled operators have been fused, with their input now added to F), and obtaining a final data streaming computation graph based on a non-fused second operator in the target data streaming computation graph and the third operator (a final data streaming graph is shown within the right box, the operator ending in AOut0 is the third operator, and the operator ending in F is a non-fused second operator).
PNG
media_image1.png
830
1127
media_image1.png
Greyscale
The recited feature does not explicitly mention granularity, however Examiner notes that Fengbin states: (Fengbin [0109]) “As shown in FIG. 9 , by saving the previous calculation result C, A+=AIN0, C=A*B can be modified to C+==AIN0*B…It can be seen that C can be obtained by calculating only once each time, which reduces the amount of calculation and improves the calculation efficiency”, so Fengbin does disclose that the fusing of operators can increase operator granularity to improve computation efficiency.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Fengbin et al. (Chinese Patent Application Publication No. 106547522), hereinafter Fengbin, further in view of Nicol (U.S. Patent Application Publication No. 2019/0279038), hereinafter Nicol.
Regarding claim 1,
Fengbin teaches A computer-implemented method [for adjusting a deep learning network] on a computer comprising a processor, the method comprising: ((Fengbin [0211]) “Software devices (or simply ‘software’) mainly include general-purpose processors (such as CPUs) and some supporting devices (such as memory, hard disks and other storage devices). The processors can be programmed to have the processing functions of stream application optimization in the embodiments of the present invention”, Fengbin does not teach adjusting a deep learning network)
acquiring, using the processor, an initial data streaming computation graph that comprises first operators for computing initial constant expressions; ((Fengbin [0177]) “An acquisition module 201 is used to acquire an application flow graph and an initial operator code of a stream application to be optimized, wherein the application flow graph includes operators, and the initial operator code is used to describe the initial operation logic carried by the operator”, an application flow graph with initial operator code corresponds to an initial data streaming computation graph, initial operation logic carried by operators corresponds to operators for computing initial constant expressions)
and obtaining, using the processor, a target data streaming computation graph according to parameters in the initial constant expressions, ((Fengbin [0133]) “Whether to migrate the second operation logic to operator M or the first operation logic to operator N can be determined based on the complexity of operator M and operator N. As shown in Figure 12, the complexity of operator N is much higher than the complexity of operator M. Therefore, the second operation logic can be migrated to operator M to obtain an optimized application data flow graph”, Fengbin Fig. 12 shows that migration is done based on parameters such as AOut0 and BIN0 in the constant expressions, an optimized application data flow graph corresponds to a target data streaming computation graph)
PNG
media_image1.png
830
1127
media_image1.png
Greyscale
wherein the target data streaming computation graph comprises a second operator ((Fengbin [0133]) “Whether to migrate the second operation logic to operator M or the first operation logic to operator N can be determined based on the complexity of operator M and operator N. As shown in Figure 12, the complexity of operator N is much higher than the complexity of operator M. Therefore, the second operation logic can be migrated to operator M to obtain an optimized application data flow graph”, Fengbin Fig. 12 shows that the optimized application data flow graph contains numerous operators, such as +=, *, +, and =, an optimized application data flow graph corresponds to a target data streaming computation graph) and a granularity of the second operator is larger than a granularity of one of the first operators ((Fengbin [0025]) “the second optimization module is specifically used to: determine the logical operation complexity of the upstream operator and the downstream operator; when the logical operation complexity of the upstream operator is higher than the logical operation complexity of the downstream operator, migrate the first operation logic to the downstream operator”, the upstream operator corresponds to the first operator, the downstream operator corresponds to the second operator, logical operation complexity corresponds to granularity) to enable an adjustment of an amount of computation [of a deep learning acceleration chip;] ((Fengbin [0109]) “As shown in FIG. 9 , by saving the previous calculation result C, A+=AIN0, C=A*B can be modified to C+==AIN0*B…It can be seen that C can be obtained by calculating only once each time, which reduces the amount of calculation and improves the calculation efficiency”, a deep learning acceleration chip is not taught by Fengbin)
wherein the second operator is used for computing a target expression obtained based on the parameters in the initial constant expressions; ((Fengbin [0109]) “by saving the previous calculation result C, A+=AIN0, C=A*B can be modified to C+==AIN0*B”, C+==AIN0*B is a target expression based on parameters originally in expressions A+=AIN0 and C=A*B and contains a second operator +==)
wherein a plurality of target expressions and a plurality of second operators are configured; ((Fengbin [0133]) “As shown in Figure 12, the complexity of operator N is much higher than the complexity of operator M. Therefore, the second operation logic can be migrated to operator M to obtain an optimized application data flow graph”, Fengbin Fig. 12 shows that the optimized application data flow graph contains numerous operators and expressions such as C+==AIN0*B (shown as a subgraph of the nodes leading up to node C), an optimized application data flow graph corresponds to a target data streaming computation graph)
and after obtaining the target data streaming computation graph according to the parameters in the initial constant expressions, the method further comprises: acquiring, using the processor, from among the plurality of second operators, at least two second operators for computing a same target expression; (Fengbin Fig. 12 shows that the process of creating the optimized application data flow graph selects operators on AOut0 and D*E that compute equivalent operations)
fusing, using the processor, the at least two second operators to obtain a third operator; (Fengbin Fig. 12 shows that the optimized application data flow graph has fused operators on AOut0 and D*E into one operator)
and obtaining, using the processor, a final data streaming computation graph based on a non-fused second operator in the target data streaming computation graph and the third operator; ((Fengbin [0133]) “As shown in Figure 12, the complexity of operator N is much higher than the complexity of operator M. Therefore, the second operation logic can be migrated to operator M to obtain an optimized application data flow graph”, Fengbin Fig. 12 shows that the optimized application data flow graph contains both an operator consisting of fused operators, at BIN0, as well as non-fused operators, such as that in H+=BIN2, an optimized application data flow graph corresponds to a target data streaming computation graph)
Although Fengbin teaches a method for acquiring an application flow graph with initial operator code for a stream application, and transforming it into a target data streaming computation graph with less logical complexity (i.e. higher granularity), Fengbin does not teach controlling a deep learning acceleration chip using the graph. Nicol teaches A method for adjusting a deep learning network ((Nicol Abstract) “Techniques are disclosed for data flow graph node parallel update for machine learning…Training data is issued to the first plurality of processing elements. The training data is used to update variables within the at least one variable node”), including the following further partial limitation that Fengbin does not teach:
wherein the [final] data [streaming] computation graph is used for controlling the deep learning acceleration chip to perform data computation ((Nicol [0009]-[0010]) “A data flow graph shows operations performed on data. The data flow graph includes nodes that represent the logical, mathematical, Boolean, and other operations to be performed on data…A reconfigurable fabric can be configured or ‘coded’ to implement a given data flow graph…The reconfigurable fabric can include computational or processor elements, storage elements, switching elements for data transfer, control elements, and so on. The reconfigurable fabrics are coded to implement a variety of processing topologies for machine learning”, a reconfigurable fabric with processor and storage elements coded to implement processing for machine learning corresponds to a deep learning acceleration chip, Fengbin teaches a final data streaming computation graph)
At the time of filing, one of ordinary skill in the art would have motivation to combine Fengbin and Nicol by taking the method for acquiring an application flow graph with initial operator code for a stream application, and transforming it into a target data streaming computation graph with less logical complexity (i.e. higher granularity), taught by Fengbin, and combining it with the method of using a data flow graph to code a reconfigurable fabric for machine learning purposes, taught by Nicol, as Nicol teaches: (Nicol [0007]) “The greater the quantity of data, and the higher the quality of the data that is processed, the better the outcome of the machine learning. The processors on which the machine learning techniques can be executed are designed to efficiently handle the flow of data. These processors, which are based on data flow architectures, process data when valid data is presented to the processor. Data flow architectures enable simplifications to a processing system such as avoiding a need for a global system clock”.
Regarding claim 2,
Fengbin and Nicol jointly teach The method according to claim 1,
Fengbin further teaches:
wherein the second operator is obtained by fusion of at least two of the first operators ((Fengbin [0109]) “by saving the previous calculation result C, A+=AIN0, C=A*B can be modified to C+==AIN0*B”, C+==AIN0*B contains a second operator +== that is a fusion of the operators in the expressions A+=AIN0 and C=A*B)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Fengbin and Nicol for the parent claim of claim 2, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 5,
Fengbin and Nicol jointly teach The method according to claim 1,
Fengbin further teaches:
wherein at least one non-fused second operator in the target streaming computation graph is configured, and at least one third operator is configured; (Fengbin Fig. 12 shows that the optimized application data flow graph contains both an operator consisting of fused operators, configured to compute BIN0, as well as non-fused operators, such as that in H+=BIN2, which is configured to compute BIN2 an optimized application data flow graph corresponds to a target data streaming computation graph)
obtaining, using the processor, the final data streaming computation graph based on the non-fused second operator in the target data streaming computation graph and the third operator comprises: combining, using the processor, one of the at least one non-fused second operators and one of the at least one third operators for computing a plurality of related target expressions in the target streaming computation graph into one data path, ((Fengbin [0121]) “As shown in FIG11 , the data flow graphs of operator M and operator N are spliced together according to the direction of the data flow in the application flow graph shown in FIG3 , and the application data flow graph of the stream application is obtained”, Fengbin Fig. 11 shows that the splicing into one path occurs between the same expressions that are combined to create the optimized application data flow graph of Fig. 12, an optimized application data flow graph corresponds to a target and final data streaming computation graph)
PNG
media_image2.png
832
840
media_image2.png
Greyscale
wherein the plurality of related target expressions represent that output of an operator for computing one of the plurality of related target expressions is input of an operator for computing another one of the plurality of related target expressions; (Fengbin Fig. 11 shows that the output of the operators for computing the expression that ends in AOut0 is input for the operators that compute the expression that ends in BOut0)
and obtaining, using the processor, the final data streaming computation graph based on all data paths ((Fengbin [0133]) “As shown in Figure 12, the complexity of operator N is much higher than the complexity of operator M. Therefore, the second operation logic can be migrated to operator M to obtain an optimized application data flow graph”, Fengbin Fig. 12 shows that the optimized application data flow graph is based on multiple data paths, an optimized application data flow graph corresponds to a final data streaming computation graph)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Fengbin and Nicol for the parent claim of claim 5, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 6,
Fengbin and Nicol jointly teach The method according to claim 5,
Fengbin further teaches:
wherein the data path comprises a header operator, a successor operator and an output operator, wherein the header operator is used for undertaking initialization of all parameters, the successor operator is used for acquiring output of a predecessor operator, and the output operator is used for outputting data ((Fengbin [0132]) “As shown in Figure 12, the operation logic of operator M is the first operation logic, and part of the operation logic in operator N is the second operation logic. The output data AOut0 of operator M is to be transmitted to the entry BIN0 of operator N”, Fengbin Fig. 12 shows a data path with a header operator that initializes the parameter BIN0, a successor operator of F+BIN0=G that acquires the output of the header operator with output AOut0, and an output operator of G*H=BOut0)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Fengbin and Nicol for the parent claim of claim 6, claim 5. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 7,
Fengbin and Nicol jointly teach The method according to claim 1,
Fengbin further teaches:
wherein a granularity of the third operator is larger than the granularity of the second operator ((Fengbin [0102-0104]) “Next, when traversing according to C=A*B marked in ④, add 1 to the statistical values of node A, node B, node "*", node "=", and node C respectively. Then the statistical values of node A, node B, node "*", node "=", and node C become 3, 2, 2, 2, and 2 respectively. Specifically, the data flow diagram of the operator marked ④ in the upper left corner of Figure 8 is shown. The hotspot path can be determined by the statistical values in the data flow graph of the fourth operator, that is, the path where the statistical values of each node are the highest. After determining the hot spot path, the local code corresponding to the hot spot path can be determined. The local operator codes corresponding to the hot spot path shown in Figure 8 are A+=AIN0, C=A*B and B+=AIN1, C=A*B. The optimized operator code can be obtained by optimizing the local operator code corresponding to the hot spot path.”, granularity corresponds to the statistical values indicating a hot spot path, the third operator corresponds to the operations that comprise the hot spot path which are used into one operation)
At the time of filing, one of ordinary skill in the art would have motivation to combine the method jointly taught by Fengbin and Nicol for the parent claim of claim 7, claim 1. No new embodiments are introduced, so the reason to combine is the same as for the parent claim.
Regarding claim 9,
Claim 9 recites a server comprising a processor and storage containing a program for performing the function of the method of claim 1. Specifically, claim 9 recites A server, comprising: at least one processor; and a storage device configured to store at least one program, wherein when executed by the at least one processor, the at least one program causes the at least one processor to perform the following steps: [the method of claim 1].
Nicol recites: (Nicol [0029]) “Techniques are disclosed for data flow graph node parallel update for machine learning. Data flow graph node parallel update can be performed on a server”, with at least one processor and storage capabilities being inherent to a server.
All other limitations in claim 9 are substantially the same as those in claim 1, therefore the same rationale for rejection applies.
Regarding claim 10,
Claim 10 recites a non-transitory computer-readable storage medium containing a program for performing the function of the method of claim 1. Specifically, claim 10 recites A non-transitory computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the method for adjusting a deep learning network according to claim 1.
Fengbin recites: (Fengbin [0248]) “A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the relevant hardware through a program, and the program can be stored in a computer-readable storage medium”.
All other limitations in claim 10 are substantially the same as those in claim 1, therefore the same rationale for rejection applies.
Regarding claim 11,
Claim 11 recites a server for performing the function of the method of claim 2. All other limitations in claim 11 are substantially the same as those in claim 2, therefore the same rationale for rejection applies.
Regarding claim 14,
Claim 14 recites a server for performing the function of the method of claim 5. All other limitations in claim 14 are substantially the same as those in claim 5, therefore the same rationale for rejection applies.
Regarding claim 15,
Claim 15 recites a server for performing the function of the method of claim 6. All other limitations in claim 15 are substantially the same as those in claim 6, therefore the same rationale for rejection applies.
Regarding claim 16,
Claim 16 recites a server for performing the function of the method of claim 7. All other limitations in claim 16 are substantially the same as those in claim 7, therefore the same rationale for rejection applies.
Regarding claim 17,
Claim 17 recites a storage medium for performing the function of the method of claim 2. All other limitations in claim 17 are substantially the same as those in claim 2, therefore the same rationale for rejection applies.
Regarding claim 19,
Claim 19 recites a storage medium for performing the function of the method of claim 5. All other limitations in claim 19 are substantially the same as those in claim 5, therefore the same rationale for rejection applies.
Regarding claim 20,
Claim 20 recites a storage medium for performing the function of the method of claim 6. All other limitations in claim 20 are substantially the same as those in claim 6, therefore the same rationale for rejection applies.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Zhang et al. (U.S. Patent Application Publication No. 2017/0300367) teaches a method of disassembling a streaming graph according to a maximum atom division principle to create a subgraph, which is then subjected to adjacency operator combination in order to create an optimized streaming subgraph.
Andrade et al. (U.S. Patent No. 8,782,628) teaches a method of partitioning an operator flow graph into sets of processing elements.
Andrade et al. (U.S. Patent Application Publication No. 2010/0293535) teaches a method of compiling a data stream processing application.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to VICTOR A NAULT whose telephone number is (703) 756-5745. The examiner can normally be reached M - F, 12 - 8.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Miranda Huang can be reached at (571) 270-7092. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/V.A.N./Examiner, Art Unit 2124
/MIRANDA M HUANG/Supervisory Patent Examiner, Art Unit 2124
Prosecution Insights