DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In communications filed on 07/08/2025. Claims 1-6 are pending in this examination.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status. This examination is in response to US Patent Application No. 19/262,090.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-6 are rejected under 35 U.S.C. 103 as being unpatentable over US Patent Application No. (US US20180341642) (filed in IDS 01/05/2022) issued to Akerib ( filed in IDS 11/06/2025) and in view of US Patent No. Application (US2008/0065547) issued to Shimizu ( filed in IDS 11/06/2025).
Regarding claim 1, Akerib discloses a system comprising: a secure, in-memory unit implemented on an associative processing unit (APU), for performing a secure similarity search, said unit to implement:
[¶10, There is therefore provided, in accordance with a preferred embodiment of the present invention, a system for natural language processing. The system includes a memory array and an in-memory processor. The memory array has rows and columns and is divided into a similarity section initially storing a plurality of feature or key vectors, a SoftMax section in which to determine probabilities of occurrence of the feature or key vectors, a value section initially storing a plurality of modified feature vectors, and a marker section. Operations in one or more columns of the memory array are associated with one feature vector to be processed], and [¶51, Reference is now made to FIGS. 1A and 1B, which are schematic illustrations of a memory computation device 100, constructed and operative in accordance with a preferred embodiment of the present invention. As illustrated in FIG. 1A, device 100 may comprise a memory array 110 to store a dataset, a k-Mins processor 120, implemented on a memory logic element, to perform a k-Mins operation and a k-Mins temporary store 130 that may be used for storing intermediate and final results of operations made by k-Mins processor 120 on data stored in memory array 110. In FIG. 1B the physical aspects of k-Mins processor 120 and the k-Mins temporary store 130 are illustrated in associative memory array 140. Associative memory array 140 combines the operations of k-Mins processor 120 and the store of k-Mins temporary store 130. Memory array 110 may store a very large dataset of binary numbers. Each binary number is comprised of a fixed number of bits and is stored in a different column in memory array 110. K-Mins temporary store 120 may store copies of the information stored in memory array 110 and several vectors storing temporary information related to a step of the computation performed by k-Mins processor 120 as well as the final result including an indication of k columns storing the k lowest values in the dataset], and [ ¶115, Applicant has realized that an associative processing unit (APU), such as memory computation device 100, can do any function of search, such as cosine similarity which is not an exact match, to achieve all that is needed for natural language processing with a neural network]; and
an encoded vector data store to store a plurality of encoded search candidate vectors
[ Abstract, a system for natural language processing includes a memory array and a processor. The memory array is divided into a similarity section storing a plurality of feature vectors, a SoftMax section in which to determine probabilities of occurrence of the feature vectors, a value section storing a plurality of modified feature vectors, and a marker section. The processor activates the array to perform parallel operations in each column indicated by the marker section: a similarity operation in the similarity section between a vector question and feature vectors stored in indicated columns; a SoftMax operation in the SoftMax section to determine an associated SoftMax probability value for indicated feature vectors; a multiplication operation in the value section to multiply the associated SoftMax value by modified feature vectors stored in indicated columns; and a vector sum in the value section to accumulate an attention vector of output of the multiplication operation]; and
and a similarity searcher to perform a similarity search between an encoded search query vector and said plurality of encoded search candidate vectors.
[¶10, There is therefore provided, in accordance with a preferred embodiment of the present invention, a system for natural language processing. The system includes a memory array and an in-memory processor. The memory array has rows and columns and is divided into a similarity section initially storing a plurality of feature or key vectors, a SoftMax section in which to determine probabilities of occurrence of the feature or key vectors, a value section initially storing a plurality of modified feature vectors, and a marker section. Operations in one or more columns of the memory array are associated with one feature vector to be processed], and [¶51, Reference is now made to FIGS. 1A and 1B, which are schematic illustrations of a memory computation device 100, constructed and operative in accordance with a preferred embodiment of the present invention. As illustrated in FIG. 1A, device 100 may comprise a memory array 110 to store a dataset, a k-Mins processor 120, implemented on a memory logic element, to perform a k-Mins operation and a k-Mins temporary store 130 that may be used for storing intermediate and final results of operations made by k-Mins processor 120 on data stored in memory array 110. In FIG. 1B the physical aspects of k-Mins processor 120 and the k-Mins temporary store 130 are illustrated in associative memory array 140. Associative memory array 140 combines the operations of k-Mins processor 120 and the store of k-Mins temporary store 130. Memory array 110 may store a very large dataset of binary numbers. Each binary number is comprised of a fixed number of bits and is stored in a different column in memory array 110. K-Mins temporary store 120 may store copies of the information stored in memory array 110 and several vectors storing temporary information related to a step of the computation performed by k-Mins processor 120 as well as the final result including an indication of k columns storing the k lowest values in the dataset], and [ ¶115, Applicant has realized that an associative processing unit (APU), such as memory computation device 100, can do any function of search, such as cosine similarity which is not an exact match, to achieve all that is needed for natural language processing with a neural network].
Akerib does not explicitly disclose, however, Shimizu discloses a decryptor to decrypt an encrypted, encoded vector into an encoded vector
[¶8, digital content is first encoded (or compressed) and then encrypted, which is the primary control point for content protection against piracy. To play back the content, it must be decrypted first and then decoded], and [¶30, APU 140 authenticates the received program thread, and retrieves encrypted device keys 195 from main memory 190. In turn, APU 140 decrypts encrypted device keys 195 and calculates a title key based upon the decrypted device keys. As such, APU 140 in conjunction with local store 150 decrypt (using the title key) and decode encrypted/encoded digital content. During the decryption and decoding stages, APU 140 uses physical addresses 145 to access local store 150 (not translated addresses) so that secure processing vault 130 is not compromised when a malicious client attempts to alter address translation tables. When APU 140 finishes decrypting and decoding the encrypted/encoded digital content, APU 140 provides the decrypted and decoded digital content back to MPU 110 or a graphics card for further processing], and [¶43, FIG. 4 is a high-level flowchart showing steps taken in a main processing unit requesting an attached processing complex to decrypt and decode encrypted/encoded digital content. A main processing unit (MPU), such as MPU 110 shown in FIG. 1, passes program threads corresponding to encrypted/encoded digital content to an attached processing unit (APU), such as APU 140 shown in FIG. 1. In turn, the APU decrypts and decodes the data within a secure processing vault and passes decrypted and decoded digital content back to the main processing unit for further processing. For example, the digital content may be a digital video stream, in which case the main processing unit formats the digital video stream for viewing on a monitor. In another example, the digital content may be a digital audio stream, in which case the main processing unit formats the digital audio stream for playing on speakers].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teaching of Akerib, with the teaching of Shimizu in order to implement a system and method for decrypting and decoding encrypted/encoded digital content within a hardware-based secure environment [ Shimizu, ¶2].
Regarding claim 2, Akerib discloses wherein said encoded vector is one of: an encoded search query vector and an encoded search candidate vector.
[ Abstract, a system for natural language processing includes a memory array and a processor. The memory array is divided into a similarity section storing a plurality of feature vectors, a SoftMax section in which to determine probabilities of occurrence of the feature vectors, a value section storing a plurality of modified feature vectors, and a marker section. The processor activates the array to perform parallel operations in each column indicated by the marker section: a similarity operation in the similarity section between a vector question and feature vectors stored in indicated columns; a SoftMax operation in the SoftMax section to determine an associated SoftMax probability value for indicated feature vectors; a multiplication operation in the value section to multiply the associated SoftMax value by modified feature vectors stored in indicated columns; and a vector sum in the value section to accumulate an attention vector of output of the multiplication operation.
Regarding claim 3, Akerib discloses wherein said vector data store to store said encoded search candidate vectors in columns.
[ Abstract, a system for natural language processing includes a memory array and a processor. The memory array is divided into a similarity section storing a plurality of feature vectors, a SoftMax section in which to determine probabilities of occurrence of the feature vectors, a value section storing a plurality of modified feature vectors, and a marker section. The processor activates the array to perform parallel operations in each column indicated by the marker section: a similarity operation in the similarity section between a vector question and feature vectors stored in indicated columns; a SoftMax operation in the SoftMax section to determine an associated SoftMax probability value for indicated feature vectors; a multiplication operation in the value section to multiply the associated SoftMax value by modified feature vectors stored in indicated columns; and a vector sum in the value section to accumulate an attention vector of output of the multiplication operation.
[0010] There is therefore provided, in accordance with a preferred embodiment of the present invention, a system for natural language processing. The system includes a memory array and an in-memory processor. The memory array has rows and columns and is divided into a similarity section initially storing a plurality of feature or key vectors, a SoftMax section in which to determine probabilities of occurrence of the feature or key vectors, a value section initially storing a plurality of modified feature vectors, and a marker section. Operations in one or more columns of the memory array are associated with one feature vector to be processed. The in-memory processor activates the memory array to perform the following operations in parallel in each column indicated by the marker section.
Regarding claim 4, Akerib discloses said similarity searcher to perform said similarity search of said plurality of encoded search candidate vectors in said columns in a parallel process.
[ Abstract, a system for natural language processing includes a memory array and a processor. The memory array is divided into a similarity section storing a plurality of feature vectors, a SoftMax section in which to determine probabilities of occurrence of the feature vectors, a value section storing a plurality of modified feature vectors, and a marker section. The processor activates the array to perform parallel operations in each column indicated by the marker section: a similarity operation in the similarity section between a vector question and feature vectors stored in indicated columns; a SoftMax operation in the SoftMax section to determine an associated SoftMax probability value for indicated feature vectors; a multiplication operation in the value section to multiply the associated SoftMax value by modified feature vectors stored in indicated columns; and a vector sum in the value section to accumulate an attention vector of output of the multiplication operation.
[0010] There is therefore provided, in accordance with a preferred embodiment of the present invention, a system for natural language processing. The system includes a memory array and an in-memory processor. The memory array has rows and columns and is divided into a similarity section initially storing a plurality of feature or key vectors, a SoftMax section in which to determine probabilities of occurrence of the feature or key vectors, a value section initially storing a plurality of modified feature vectors, and a marker section. Operations in one or more columns of the memory array are associated with one feature vector to be processed. The in-memory processor activates the memory array to perform the following operations in parallel in each column indicated by the marker section.
Regarding claim 5, Akerib discloses the system of claim 1wherein said similarity search is a nearest neighbor search.
[0097] The k-Mins algorithm described hereinabove may be used by the k nearest neighbors (K-NN) data mining algorithm. In K-NN D may represent a large dataset containing q objects (q enormously large). D.sup.P is one object in dataset D: D.sup.P E D and A is the object to classify. An object is defined by a vector of numerical attributes: A is defined by a vector [A.sub.0, A.sub.1, . . . A.sub.n] of n attributes and D.sup.P is defined by a vector [D.sub.1.sup.p, D.sub.2.sup.p, D.sub.3.sup.p, . . . D.sub.n.sup.p] of the same n attributes. A distance, which is a binary number C.sup.P of m bits, between object A and object D.sup.P, is calculated between the introduced object A and each object D.sup.P in the dataset D. The distance C.sup.P may represent the cosine similarity between two non-zero vectors. The known in the art cosine similarity associates each pair of vectors with a scalar quantity and is known as the inner product of the vectors.
Regarding claim 6, Akerib discloses wherein said APU implemented on one of: SRAM, non-volatile, and non-destructive memory.
[¶124, Memory computation device 100 may be formed of any suitable memory array, such as an SRAM, a non-volatile, a volatile, and a non-destructive array and may be formed into a plurality of bit line processors 114, each processing one bit of a word and each word being stored in a column of associative memory array 140].
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's
disclosure.
See submitted 892 for more relevant references.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SHAHRIAR ZARRINEH whose telephone number is (571)272-1207. The examiner can normally be reached Monday-Friday, 8:30am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jorge Ortiz-Criado can be reached at 571-272-7624. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SHAHRIAR ZARRINEH/Primary Examiner, Art Unit 2496
Prosecution Insights