CTNF 18/488,454 CTNF 92914 DETAILED ACTION Notice of Pre-AIA or AIA Status 07-03-aia AIA 15-10-aia The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA. This action is responsive to the Application filed on October 17, 2023. Claims 1-20 are pending in the case. Claims 1, 9, and 17 are the independent claims. This action is non-final. Claim Rejections – 35 USC § 102 07-06 AIA 15-10-15 In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. 07-07-aia AIA 07-07 The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action: A person shall be entitled to a patent unless – 07-08-aia AIA (a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention. 07-12-aia AIA (a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention. 07-15 AIA Claim s 1-4, 8-12, and 16-20 are rejected under 35 U.S.C. 102( a)(1 ) as being anticipated by Luciw et al. (US 20180330238 A1) . With respect to claims 1, 9, and 17, Luciw teaches a collective communication system, wherein the system comprises at least a first computing chip and a second computing chip configured to perform a method; a computer device, comprising a processor, a memory, a first computing chip, and a second computing chip, wherein the memory is configured to store computer-executable instructions, and the processor executes the computer-executable instructions in the memory, the computer-executable instructions, when executed, cause the first computing chip and the second computing chip perform the operations of the method ( e.g. paragraph 0047, Fig. 1, L-DNN architecture where multiple devices including master edge/central server and several compute edges operate in concert; each device connecting to central server to meld (fuse, merge, or combine) newly acquired knowledge and share with other L-DNNs; paragraph 0051, modules A and B (i.e. of compute edges and master edge/central server as shown in Fig. 1) implemented in suitable processors with appropriate volatile and non-volatile memory; paragraph 0149 and 0153, described embodiments implemented using combination of hardware and software, i.e. software code executed on processors ); and a collective communication method, wherein method comprises: compressing, by a first computing chip, first data ( e.g. paragraph 0097, real-time operating machines learning new knowledge on the edge; compressed generalized representation (i.e. of the new knowledge, represented in terms of a synaptic weight matrix, to be sent from the edge to the central server, such that the edge, using its corresponding computing processor/chip, compresses the new knowledge/first data) ); sending, by the first computing chip, compressed first data to a second computing chip through a communication channel ( e.g. paragraph 0097, in order to share the new knowledge, each real-time operating machine sends compressed generalized representation from the edge to a central server; paragraph 0100, real-time operating machine transmitting consolidated weight matrix to central server through a wired or wireless communication channel; i.e. the edge device, using its corresponding processor/chip, sends the compressed new knowledge/first data via a corresponding communication channel to a central sever and its corresponding processor/chip ); and performing, by the second computing chip, an operation based on the compressed first data ( e.g. paragraph 0097, knowledge acquired by real-time operating machine extracted, appended, and consolidated in central server; paragraph 0099, describing consolidation process in which compressed representations of the new objects are integrated with representations of previously known objects; paragraph 0101, after weight matrices from real-time operating machines are collected at the central server, the central server runs melding utility that combines, compresses, and consolidates the knowledge that is acquired from each real-time operating machine into a single weight matrix ). With respect to clams 2, 10, and 18, Luciw teaches all of the limitations of claims 1, 9, and 17 as previously discussed, and further teaches wherein the second computing chip is a root node of the communication channel ( e.g. paragraphs 0047, 0097, and 0099-0101 as cited above, master edge/central server ), and the performing, by the second computing chip, an operation based on the compressed first data comprises: aggregating, by the second computing chip, the first data with compressed second data, wherein the second data is to-be-communicated data on the second computing chip ( e.g. paragraph 0097, knowledge acquired by real-time operating machine extracted, appended, and consolidated in central server; paragraph 0099, describing consolidation process in which compressed representations of the new objects are integrated with representations of previously known objects; paragraph 0101, after weight matrices from real-time operating machines are collected at the central server, the central server runs melding utility that combines, compresses, and consolidates the knowledge that is acquired from each real-time operating machine into a single weight matrix ); and sending, by the second computing chip, an aggregation result to the first computing chip ( e.g. paragraph 0102, resulting weight matrix created during melding is downloaded to the real-time operating machines through communication channels; paragraph 0130, pushing merged information from central server back to all drones; paragraph 0133, new knowledge integrated centrally is pushed back after melding to each individual device ). With respect to claims 3, 11, and 19, Luciw teaches all of the limitations of claims 1, 9, and 17 as previously discussed, and further teaches wherein the second computing chip is a root node of the communication channel, and the performing, by the second computing chip, an operation based on the compressed first data comprises: combining, by the second computing chip, the first data with compressed second data, wherein the second data is to-be-communicated data on the second computing chip ( e.g. paragraph 0097, knowledge acquired by real-time operating machine extracted, appended, and consolidated in central server; paragraph 0099, describing consolidation process in which compressed representations of the new objects are integrated with representations of previously known objects; paragraph 0101, after weight matrices from real-time operating machines are collected at the central server, the central server runs melding utility that combines, compresses, and consolidates the knowledge that is acquired from each real-time operating machine into a single weight matrix ); and sending, by the second computing chip, a combination result to the first computing chip ( e.g. paragraph 0102, resulting weight matrix created during melding is downloaded to the real-time operating machines through communication channels; paragraph 0130, pushing merged information from central server back to all drones; paragraph 0133, new knowledge integrated centrally is pushed back after melding to each individual device ). With respect to claims 4, 12, and 20, Luciw teaches all of the limitations of claims 1, 9, and 17 as previously discussed, and further teaches wherein the first computing chip communicates with the second computing chip through a plurality of communication channels, the plurality of communication channels comprise a first communication channel ( e.g. paragraph 0102, transmission of data between real-time operating machines and central server through wired or wireless communication channels; paragraph 0152, computers interconnected by one or more networks in any suitable form; i.e. the central server/master edge and real-time device/compute edge may communicate through a plurality of communication channels/networks, including at least a first channel/network ), and the sending, by the first computing chip, compressed first data to the second computing chip through the communication channel comprises: sending, by the first computing chip, a first part of the first data to the second computing chip through the first communication channel, wherein the second computing chip is a root node of the first communication channel ( e.g. paragraph 0097, in order to share the new knowledge, each real-time operating machine sends compressed generalized representation from the edge to a central server; paragraph 0100, real-time operating machine transmitting consolidated weight matrix to central server through a wired or wireless communication channel; i.e. the edge device, using its corresponding processor/chip, sends the compressed new knowledge/first data, analogous to at least a first part of the first data, via a corresponding communication channel to a central sever, analogous to a root node, and its corresponding processor/chip ); and the performing, by the second computing chip, an operation based on the compressed first data specifically comprises: aggregating, by the second computing chip, a part of the compressed first data with a part of second data, wherein the second data is to-be-communicated data on the second computing chip ( e.g. paragraph 0097, knowledge acquired by real-time operating machine extracted, appended, and consolidated in central server; paragraph 0099, describing consolidation process in which compressed representations of the new objects are integrated with representations of previously known objects; paragraph 0101, after weight matrices from real-time operating machines are collected at the central server, the central server runs melding utility that combines, compresses, and consolidates the knowledge that is acquired from each real-time operating machine into a single weight matrix; paragraph 0102, resulting weight matrix created during melding is downloaded to the real-time operating machines through communication channels; paragraph 0130, pushing merged information from central server back to all drones; paragraph 0133, new knowledge integrated centrally is pushed back after melding to each individual device; i.e. the central server/root node, via its corresponding chip/processor, aggregates at least a part of the compressed data received from the compute edge device with at least a part of additional/second data, which is to be communicated back to the compute edge device ). With respect to claims 8 and 16, Luciw teaches all of the limitations of claims 1 and 9 as previously discussed, and further teaches wherein the first computing chip or the second computing chip comprises one or more of a graphics processor, a tensor processor, a neural network processor, or a deep learning processor ( e.g. paragraph 0051, indicating that modules A and B (i.e. of compute edges and master edge/central server as shown in Fig. 1) can be implemented in suitable processors such as graphics processor units, etc. ) . Claim Rejections – 35 USC § 103 07-20-aia AIA The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action: A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made. 07-23-aia AIA The factual inquiries set forth in Graham v. John Deere Co. , 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows: 1. Determining the scope and contents of the prior art. 2. Ascertaining the differences between the prior art and the claims at issue. 3. Resolving the level of ordinary skill in the pertinent art. 4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 07-21-aia AIA Claim s 5, 6, 13, and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Luciw in view of Misawa et al. (US 20210118035 A1) . With respect to claims 5 and 13, Luciw teaches all of the limitations of claims 1 and 9 as previously discussed, and further teaches wherein the first computing chip and the second computing chip is included in a collective communication system ( e.g. paragraph 0047, Fig. 1, L-DNN architecture where multiple devices including master edge/central server and several compute edges operate in concert; each device connecting to central server to meld (fuse, merge, or combine) newly acquired knowledge and share with other L-DNNs ). Luciw does not explicitly disclose the collective communication system is configured to recommend merchandise to a user by using a recommendation model with reference to a user feature and a merchandise feature, and before the compressing, by the first computing chip, first data, the method comprises: converting, by the first computing chip, the user feature and the merchandise feature into the first data based on an embedding table; and the method further comprises: inputting, by the second computing chip into the recommendation model, an operation result obtained by the second computing chip by performing an operation based on the compressed first data, to obtain an updated value of the embedding table and an updated value of the recommendation model; updating, by the second computing chip, the recommendation model based on the updated value of the recommendation model; and updating, by the second computing chip, the embedding table based on the updated value of the embedding table. However, Misawa teaches the collective communication system is configured to recommend merchandise to a user by using a recommendation model with reference to a user feature and a merchandise feature ( e.g. paragraph 0005, model that has learned the correlation between feature of user data and feature of product data and recommending means for acquiring feature data of user data and features of the product data, calculating matching degree using machine learning to which the model is applied, and determining a recommendation ranking of the product; paragraph 0071, outputting product likely to be purchased together with recommended product based on history information of user ), and before the compressing, by the first computing chip, first data, the method comprises: converting, by the first computing chip, the user feature and the merchandise feature into the first data based on an embedding table ( e.g. paragraph 0030, history information table that manages history information, including information about user, product, etc. as shown in Fig. 2; paragraphs 0047-0048, obtaining user/product history information using history information table; paragraph 0050, acquiring user information, product information, and advertising slogan from history information and extracting features, using method for compressing content into predetermined dimension, such as dimensional compression by an embedding layer; the feature of each of the user information, the product information, and the advertising slogan is represented by a real value vector in a predetermined dimension ); and the method further comprises: inputting, by the second computing chip into the recommendation model, an operation result obtained by the second computing chip by performing an operation based on the compressed first data, to obtain an updated value of the embedding table and an updated value of the recommendation model ( e.g. paragraph 0074, performing learning again using learning data newly created based on history information; learning based on new learning data not used in previous learning of the learning model; paragraph 0152, collecting from history information table all pieces of history information including history information newly added after learning of the learning model to create learning model, and causing the learning model to learn from the beginning; paragraph 0154, collecting history information newly added after learning of the learning model to create learning data and cause the learning model to learn again using the created learning data; causing current learning model to perform additional learning using learning data created from history information within specific period; i.e. the system may obtain additional, newly added user history information and add/combine it with existing user history information in the history information table, analogous to an operation result obtained by performing an operation based on compressed first data (i.e. combining the new data with the first compressed data), where this results in an updated value of the embedding table (i.e. the updated history information table including the new information) and then input this updated history data into the learning/recommendation model to retrain the model, resulting in an updated value of the recommendation model ); updating, by the second computing chip, the recommendation model based on the updated value of the recommendation model ( e.g. paragraphs 0074, 0152, and 0154 as cited above, performing the learning/training of the model again using the updated history information, resulting in an updated model based on updated values ); and updating, by the second computing chip, the embedding table based on the updated value of the embedding table ( e.g. paragraphs 0074, 0152, and 0154 as cited above, obtaining newly added history information and adding it to the existing history information, which is stored in the history information table, analogous to updating the embedding table based on the updated value of the embedding table ). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Luciw and Misawa in front of him to have modified the teachings of Luciw (directed to enabling continual memory-bounded learning in AI and deep learning continuously operating applications across networked compute edges), to incorporate the teachings of Misawa (directed to learning models for recommending products to users) to include the capabilities to utilize the system (i.e. of Luciw) in an environment for providing product recommendations to the user, by converting user and product features into the first data using a table, inputting the result of the operation based on the compressed first data into the recommendation model to obtain updated embedding table and recommendation model values, and update the recommendation model and embedding table using the respective updated values (as taught by Misawa). One of ordinary skill would have been motivated to perform such a modification in order to prompt a user’s buying motivation using advertising text associated with a product which is generated for that specific user as described in Misawa (paragraphs 0006-0009). With respect to claims 6 and 14, Luciw teaches all of the limitations of claims 1 and 9 as previously discussed, and further teaches wherein the first computing chip and the second computing chip is included in a collective communication system ( e.g. paragraph 0047, Fig. 1, L-DNN architecture where multiple devices including master edge/central server and several compute edges operate in concert; each device connecting to central server to meld (fuse, merge, or combine) newly acquired knowledge and share with other L-DNNs ). Luciw does not explicitly disclose the collective communication system is configured to recommend merchandise to a user by using a recommendation model with reference to a user feature and a merchandise feature, and before the compressing, by the first computing chip, first data, the method comprises: converting, by the first computing chip, the user feature and the merchandise feature into fourth data based on an embedding table; and inputting, by the second computing chip, the fourth data into the recommendation model, to obtain the first data and an updated value of the recommendation model; and the method further comprises: updating, by the second computing chip, the recommendation model based on the updated value of the recommendation model; and updating, by the second computing chip, the embedding table based on an operation result obtained by the second computing chip by performing an operation based on the compressed first data. However, Misawa teaches the collective communication system is configured to recommend merchandise to a user by using a recommendation model with reference to a user feature and a merchandise feature ( e.g. paragraph 0005, model that has learned the correlation between feature of user data and feature of product data and recommending means for acquiring feature data of user data and features of the product data, calculating matching degree using machine learning to which the model is applied, and determining a recommendation ranking of the product; paragraph 0071, outputting product likely to be purchased together with recommended product based on history information of user ), and before the compressing, by the first computing chip, first data, the method comprises: converting, by the first computing chip, the user feature and the merchandise feature into fourth data based on an embedding table ( e.g. paragraph 0030, history information table that manages history information, including information about user, product, etc. as shown in Fig. 2; paragraphs 0047-0048, obtaining user/product history information using history information table; paragraph 0050, acquiring user information, product information, and advertising slogan from history information and extracting features, using method for compressing content into predetermined dimension, such as dimensional compression by an embedding layer; the feature of each of the user information, the product information, and the advertising slogan is represented by a real value vector in a predetermined dimension ); and inputting, by the second computing chip, the fourth data into the recommendation model, to obtain the first data and an updated value of the recommendation model ( e.g. paragraph 0074, performing learning again using learning data newly created based on history information; learning based on new learning data not used in previous learning of the learning model; paragraph 0152, collecting from history information table all pieces of history information including history information newly added after learning of the learning model to create learning model, and causing the learning model to learn from the beginning; paragraph 0154, collecting history information newly added after learning of the learning model to create learning data and cause the learning model to learn again using the created learning data; causing current learning model to perform additional learning using learning data created from history information within specific period; i.e. the system may obtain additional, newly added user history information, or user history information from a given specific period, and add/combine it with existing user history information in the history information table, where this results in an updated value of the embedding table (i.e. the updated history information table including the new information) and then input this updated history data into the learning/recommendation model to retrain the model, resulting in an updated value of the recommendation model and first data (i.e. updated history table information) ); and the method further comprises: updating, by the second computing chip, the recommendation model based on the updated value of the recommendation model ( e.g. paragraphs 0074, 0152, and 0154 as cited above, performing the learning/training of the model again using the updated history information, resulting in an updated model based on updated values ); and updating, by the second computing chip, the embedding table based on an operation result obtained by the second computing chip by performing an operation based on the compressed first data ( e.g. paragraphs 0074, 0152, and 0154 as cited above, obtaining newly added history information and adding it to the existing history information, which is stored in the history information table, analogous to updating the embedding table based on an operation result obtained by performing an operation based on compressed first data, i.e. adding new history information to the existing history information in the table ). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Luciw and Misawa in front of him to have modified the teachings of Luciw (directed to enabling continual memory-bounded learning in AI and deep learning continuously operating applications across networked compute edges), to incorporate the teachings of Misawa (directed to learning models for recommending products to users) to include the capabilities to utilize the system (i.e. of Luciw) in an environment for providing product recommendations to the user, by converting user and product features into the data using a table (e.g., at an edge device of Luciw), inputting the results into the recommendation model (e.g. at a central server of Luciw) to obtain updated/additional data and recommendation model values, and update (e.g. at the central server of Luciw) the recommendation model and embedding table using the respective updated values (as taught by Misawa). One of ordinary skill would have been motivated to perform such a modification in order to prompt a user’s buying motivation using advertising text associated with a product which is generated for that specific user as described in Misawa (paragraphs 0006-0009) . 07-21-aia AIA Claim s 7 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Luciw in view of Maiyuran et al. (US 20210073318 A1) . With respect to claims 7 and 15, Luciw teaches all of the limitations of claims 1 and 9 as previously discussed. Luciw does not explicitly disclose wherein an amount of data whose value is 0 is greater than an amount of data whose value is not 0 in the first data. However, Maiyuran teaches wherein an amount of data whose value is 0 is greater than an amount of data whose value is not 0 in the first data ( e.g. paragraph 0002, indicating that machine learning applications typically implement matrix multiplication workloads having significant percentage of zeros, i.e. sparse matrices; paragraph 0248, performing optimizations upon detecting significant percentage of zero elements in input matrices; paragraph 0249, compression of sparse matrix data for transmission, such as by removing zero elements and packing non-zero elements; paragraph 0255, determining matrix has adjacent rows with predetermined threshold, i.e. 50% or more, of zeros; paragraph 0256, rows having more than 50% zeros ). Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention having the teachings of Luciw and Maiyuran in front of him to have modified the teachings of Luciw (directed to enabling continual memory-bounded learning in AI and deep learning continuously operating applications across networked compute edges), to incorporate the teachings of Maiyuran (directed to sparse matrix multiplication acceleration, such as for machine learning) to include the capability to compress the first data (i.e. the weight matrix of Luciw), including in instances in which it is a sparse matrix (i.e. having a significant percentage of zeros, such as at least some relative amount of data in the matrix having more than 50 percent zeros as taught by Maiyuran). One of ordinary skill would have been motivated to perform such a modification in order to remove unnecessary operations in matrices having a large magnitude of zero values, improving performance, as described in Maiyuran (paragraph 0029). It is noted that any citation to specific pages, columns, lines, or figures in the prior art references and any interpretation of the references should not be considered to be limiting in any way. “The use of patents as references is not limited to what the patentees describe as their own inventions or to the problems with which they are concerned. They are part of the literature of the art, relevant for all they contain,” In re Heck, 699 F.2d 1331, 1332-33, 216 USPQ 1038, 1039 (Fed. Cir. 1983) (quoting in re Lemelson, 397 F.2d 1006, 1009, 158 USPQ 275, 277 (GCPA 1968)). Further, a reference may be relied upon for all that it would have reasonably suggested to one having ordinary skill the art, including nonpreferred embodiments. Merck & Co, v. Biocraft Laboratories, 874 F.2d 804, 10 USPQ2d 1843 (Fed. Cir.), cert, denied, 493 U.S. 975 (1989). See also Upsher-Smith Labs. v. Pamlab, LLC, 412 F,3d 1319, 1323, 75 USPQ2d 1213, 1215 (Fed. Cir, 2005): Celeritas Technologies Ltd. v. Rockwell International Corp., 150 F.3d 1354, 1361, 47 USPQ2d 1516, 1522-23 (Fed. Cir. 1998) . Conclusion 07-96 AIA The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure. Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEREMY L STANLEY whose telephone number is (469)295-9105. The examiner can normally be reached on Monday-Friday from 9:00 AM to 5:00 PM CST. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Al Kawsar, can be reached at telephone number (571) 270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR for authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form. /JEREMY L STANLEY/ Primary Examiner, Art Unit 2127 Application/Control Number: 18/488,454 Page 2 Art Unit: 2127
Prosecution Insights