Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 5/29/2026 has been entered.
Response to Arguments
Applicant’s arguments with respect to claims 18-31, 36 and 38-41 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
Claims 18-31, 36 and 38-41 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
In Claim 18, Applicant claims a “first circuit” and a “second circuit”, there is no support for this idea in the specification. Applicant’s specification, likewise, doesn’t disclose the now-claimed “processor” in the second circuit.
In claim 40, Applicant recites “the other of the first information signal”. There is no the other of the first information signal in the specification.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
Claim 40 recites the limitation "the other of the first information signal ". There is insufficient antecedent basis for this limitation in the claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 18-31, 40 and 41 are rejected under 35 U.S.C. 103 as being unpatentable over US20180157916A1 to Doumbouya et al and US20190114534A1 to Teng et al.
Claim 36 is rejected under 35 U.S.C. 103 as being unpatentable over US20180157916A1 to Doumbouya et al, US20190114534A1 to Teng et al and US20190228236A1 and Schlicht et al.
Claims 38 and 39 are rejected under 35 U.S.C. 103 as being unpatentable over US20180157916A1 to Doumbouya et al, US20190114534A1 to Teng et al and US20200042877A1 to Whatmough et al.
Doumbouya teaches claim 18. (Currently amended) A system for processing at least a first information signal and a second information signal, the system comprising:
a signal input configured to receive at least the first information signal from a first sensor having a first signal type, and the second information signal from a second sensor having a second signal type; (Claim 39 makes it clear that these signal types can still be the same type, images. Doumbouya para 168 “depict example embodiments of layer sharing between at least two CNNs. In at least those example embodiments, a first CNN is trained to perform a first task and a second CNN is trained to perform a second task.” Doumbouya para 169 “ the CNNs are depicted as processing images and as being trained to perform feature vector generation. However, in at least some different example embodiments (not depicted), one or more of the CNNs that share layers with each other may accept non-image data, be used for different types of tasks, or both…. The CNNs may also be trained to receive non-image data and, consequently, to performs tasks other than image processing. For example, one or more of the CNNs may receive data from any suitable type of sensor, such as an audio sensor…”)
a (Doumbouya para 168 “depict example embodiments of layer sharing between at least two CNNs. In at least those example embodiments, a first CNN is trained to perform a first task and a second CNN is trained to perform a second task. The first CNN comprises a first group of layers connected in series with a second group of layers and is configured such that data for the first CNN is input to the first group of layers. The second CNN comprises the first group of layers connected in series with a third group of layers and is configured such that data for the second CNN is input to the first group of layers.” First group of layers is the claimed shared unit.)
at least two separate units of the neural network arranged in the signal flow downstream of the shared unit, wherein a first unit of the at least two separate units is configured to perform a second signal processing step of the first information signal of the first signal type, and wherein a second unit of the at least two separate units (Applicant’s second circuit isn’t disclosed and a second circuit with a processor is not disclosed. The BRI is that the second circuit and the first circuit are the same and share a processor, while the ‘separate units’ are actually just software modules. The specification states this in paragraph 66-67 “The first unit 45 a of the two (or more) separate units 45 a, 45 b is implemented in software as trainable…. The second unit 45 b of the two (or more) separate units 45 a, 45 b is implemented in software as non-trainable.” Further, spec. 69, makes it clear that the CNN may be implemented in “hardware” if it is “e.g. non-trainable”. That shows that all these separate units are executed by one piece of hardware/circuit. This is bolstered by the purpose of this invention is “reducing the number of control units and [sic] through hardware implementation of convolutional layers.” Spec. 73. Another interpretation does not make sense in the context of Applicant’s specification. Doumbouya para 168 “depict example embodiments of layer sharing between at least two CNNs. In at least those example embodiments, a first CNN is trained to perform a first task and a second CNN is trained to perform a second task. The first CNN comprises a first group of layers connected in series with a second group of layers and is configured such that data for the first CNN is input to the first group of layers. The second CNN comprises the first group of layers connected in series with a third group of layers and is configured such that data for the second CNN is input to the first group of layers.” Second group of layers and the third group of layers are the two separate units. Doumbouya para 169 teaches that the separate units can be directed to different types of signals, “ In at least some example embodiments, the CNNs that share layers may be trained to receive different types of data (e.g., one CNN may be trained to receive audio data while the other is trained to receive image data)…”)
a signal output configured to output the processed information signals. (Doumbouya fig. 11b different output vectors are the processed information signals.)
Doumbouya doesn’t teach separate first and second circuits.
However, Teng teaches a first circuit… (Teng fig. 6 first processor 602 and accerlator 238.) and a second circuit. (Teng fig 6 second processor 604. Teng para 56 “The neural network accelerator reads (4) the input data set 610 from the RAM 608 and performs the specified subset of neural network operations. In an exemplary implementation of a convolutional neural network, the neural network accelerator performs the operations of convolutional layers, ReLU, and max-pooling layers. The neural network accelerator can also perform the operations of the fully-connected layers. However, for improved performance, the second processor element 604 performs the operations of the fully-connected layers.”)
Teng, the claims, and Doumbouya all do signal processing with a CNN. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to implement Doumbouya’s architecture in the hardware of Teng because “software that initiates the hardware accelerator, the performance improvement provided by the hardware accelerator should be greater than the processing overhead involved in moving data between the host computer system and the hardware accelerator.” Teng para 5.
Doumbouya teaches claim 19. (Currently amended) The system as claimed in claim 18, wherein the first unit of the at least two separate units of the neural network comprises at least one at least one hardware-implemented unit of the neural network. (Doumbouya para 81 “various example embodiments may take the form of an entirely hardware embodiment…”)
Doumbouya teaches claim 20. (Previously presented) The system as claimed in claim 19, wherein the at least two separate units of the neural network are disposed in parallel in the signal flow. (Doumbouya fig. 11b below)
PNG
media_image1.png
644
432
media_image1.png
Greyscale
Doumbouya teaches claim 21. (Previously presented) The system as claimed in claim 18, wherein the at least two separate units of the neural network are disposed in parallel in the signal flow. (Doumbouya fig. 11b below)
PNG
media_image1.png
644
432
media_image1.png
Greyscale
Doumbouya teaches claim 22. (Currently amended) The system as claimed in claim 18, wherein the shared unit of the neural network is hardware-implemented and includes a hardware accelerator. (Doumbouya para 96 “when implemented in this way, a general purpose processor and one or more of a GPU and a DSP may be implemented together within the SOC.”)
Doumbouya teaches claim 23. (Currently amended) The system as claimed in claim 22, wherein the first unit of the at least two separate units of the neural network comprises at least one hardware-implemented unit of the neural network. (Doumbouya para 96 “when implemented in this way, a general purpose processor and one or more of a GPU and a DSP may be implemented together within the SOC.”)
Doumbouya teaches claim 24. (Previously presented) The system as claimed in claim 23, wherein the at least two separate units of the neural network are disposed in parallel in the signal flow. (Doumbouya fig. 11b below)
PNG
media_image1.png
644
432
media_image1.png
Greyscale
Doumbouya teaches claim (Previously presented) The system as claimed in claim 18, wherein the shared unit of the neural network comprises a convolutional neural network. (Doumbouya para 168 “depict example embodiments of layer sharing between at least two CNNs. In at least those example embodiments, a first CNN is trained to perform a first task and a second CNN is trained to perform a second task. The first CNN comprises a first group of layers connected in series…”)
Doumbouya teaches claim 26. (Previously presented) The system as claimed in claim 18, wherein the shared unit of the neural network comprises an autoencoder. (Doumbouya para 168 teaches the first layers of a CNN as the shared unit. The first layers of a CNN are called autoencoders. Doumboya para 122-123 reiterates this point, “At 302, one or more foreground visual objects in the scene represented by the image frame are detected based on the segmenting of 300. … Metadata may be further generated relating to the detected one or more foreground areas. … the location metadata may be further used to generate a bounding box (such as, for example, when encoding video or playing back video) outlining the detected foreground visual object.”
Doumbouya teaches claim 27. (Previously presented) The system as claimed in claim 18, the system further comprising:
a preprocessing unit;
wherein the preprocessing unit is arranged in the signal flow between the signal input and the shared unit of the neural network, wherein the preprocessing unit is configured to allocate a respective processing time to the first information signal and the second information signal for the first signal processing step by the shared unit of the neural network. (Doumbouya para 158 “A benefit of inputting a sub-region of the image frame to the classification module 916 is that the whole scene need not be analyzed for classification, thereby requiring less processing power. The video analytics module 224 may, for example, filter out all object types except human for further processing.” The filter is the preprocessing. The filtering is done for all the input signals.)
Doumbouya teaches claim 28. (Previously presented) The system as claimed in claim 27, wherein the preprocessing unit is configured to convert the first information signal and the second information signal to a signal standard which is adapted to the shared unit of the neural network. (Doumbouya para 175 “ the data that is input to the CNNs 1010,1020 is stored as a multidimensional array having a rank of four.”)
Doumbouya teaches claim 29. (Previously presented) The system as claimed in claim 28, wherein the first information signal and the second information signal are image signals and the preprocessing unit is configured to convert the image signals into the respective standard image signals with a predetermined frame rate and/or a predetermined image resolution. (Doumbouya para 175 “ the data that is input to the CNNs 1010,1020 is stored as a multidimensional array having a rank of four.”)
Doumbouya teaches claim 30. (Previously presented) The system as claimed in claim 29, wherein the preprocessing unit is further configured to make only a predetermined selection of information from the at least a first image signal of the image signals for the signal processing by the shared unit based on a function for which the first image signal is provided. (Doumbouya para 158 “A benefit of inputting a sub-region of the image frame to the classification module 916 is that the whole scene need not be analyzed for classification, thereby requiring less processing power. The video analytics module 224 may, for example, filter out all object types except human for further processing.” The filter is the preprocessing.)
Doumbouya teaches claim 31. (Previously presented) The system as claimed in claim 27, wherein the preprocessing unit is configured to select whether the first unit or the second unit of the neural network is used for the second signal processing step depending on the first information signal or the second information signal present at signal input. (Doumbouya para 179 “the CNNs 1010,1120 of FIG. 11B receive as input a batch of chips 404 in the four-dimensional array data structure described above. For example, the CNNs 1010,1120 may receive a batch of 100 images for processing by the first two layers 1012 a,b, in which case n=100, of which 50 are person chips 404 and 50 are head chips 404. The first two layers 1012 a,b process the entire batch of images and outputs two of the four-dimensional arrays of n=50; one of the output arrays comprises the results of processing the 50 person chips 404, while the other of the output arrays comprises the results of processing the 50 head chips 404. The array comprising the data for the 50 person chips 404 is sent to the third and fourth layers 1012 c,d of the person vector CNN 1010 for further processing, while the array comprising the data for the 50 head chips 404 is sent to the third and fourth layers 1122 a,b of the head vector CNN 1120 for further processing.”)
Doumbouya teaches claim 36. (Currently amended) A motor vehicle comprising: a system as claimed in claim 18, which is arranged in a control unit of the
at least two sensors including the first sensor and the second sensor, which are connected to the signal input of the system using an on-board power supply of the (Doumboya fig. 4 server 406 and cameras 108 in fig. 1.)
Doumbouya is not a car.
However, Schlicht teaches a motor vehicle application. (Schlicht figs. 1-4)
Doumbouya, Schlicht and the claims all use a CNN to process images. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to use shared layers for a video application in a car because “repeated image evaluation in parallel paths is in parts redundant, energy-intensive and requires an increased hardware involvement.” Schlicht para 52.
Doumbouya teaches claim 38. (Previously presented) The system as claimed in claim 18, wherein the shared unit of the neural network is a (Doumbouya para 96 “when implemented in this way, a general purpose processor and one or more of a GPU and a DSP may be implemented together within the SOC.” DSP is a digital signal processor.)
Doumbouya doesn’t teach non-trainable hardware.
However, Whatmough teaches non-trainable signal processor. (Whatmough fig. 3 fixed layers 352. Whatmough para 36 “the most common approach is actually to reuse the convolutional layers from a network trained on similar data, and update the fully-connected layers using the new dataset. In this case, the weights for the “hard-wired” layers are fixed during the fine-tuning operation.”)
Whatmough, Doumbouya and the claims are all directed to CNNs. It would have been obvious to a person having ordinary skill in the art, at the time of filing, to fix layers because “[t]his has an advantage that the hard-wired layers (at least the earlier layers) can be used to provide lower-level features that will typically be useful for other tasks in related domains.” Whatmough para 36.
Teng teaches claim 39. (Previously presented) The system as claimed in claim 38, wherein the at least two separate units of the neural network comprise at least one software-implemented unit of the neural network (Teng para 28 “Thus, the neural network(s) 110 can include both hardware portions implemented in the hardware accelerator(s) 116, as well as software portions implemented in the acceleration libraries 114.”) and at least one (Teng para 28 “Thus, the neural network(s) 110 can include both hardware portions implemented in the hardware accelerator(s) 116, as well as software portions implemented in the acceleration libraries 114.”)
Teng doesn’t teach non-trainable hardware.
However, Whatmough teaches non-trainable signal processor. (Whatmough fig. 3 fixed layers 352. Whatmough para 36 “the most common approach is actually to reuse the convolutional layers from a network trained on similar data, and update the fully-connected layers using the new dataset. In this case, the weights for the “hard-wired” layers are fixed during the fine-tuning operation.”)
Teng teaches claim 40. (Currently amended) The system as claimed in claim 18, further comprising wherein one of the first information signal and the second information signal is a voice signal and the other of the first information signal and the second information signal is an image signal, and wherein the system generates image recognition information and voice recognition information. (Teng para 23 “ The inputs to each layer are data, such as images or voice samples, and trained weights, all represented as matrices.” Output from the ML framework is recognition of inputs – voice and vision.)
Doumbouya teaches claim 41. (Previously presented) The system as claimed in claim 18, wherein: the first unit is configured to perform the second signal processing step of the first information signal independent of the second information signal, and wherein the second unit is configured to perform the second signal processing step of the second information signal independent of the first information signal. (Doumbouya fig. 11 second signal processing step 1012d and 1122b performed independently on each signal, see below.)
PNG
media_image2.png
644
432
media_image2.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Austin Hicks whose telephone number is (571)270-3377. The examiner can normally be reached Monday - Thursday 8-4 PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mariela Reyes can be reached at (571) 270-1006. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AUSTIN HICKS/Primary Examiner, Art Unit 2142
Prosecution Insights