Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Drawings
The drawings are objected to as failing to comply with 37 CFR 1.84(p)(4) because reference character “1030” has been used to designate both “Sending Input of Sub-networks to Device(s)” in figure 10 and “Encoder/Decoder” in figure 14; and character “1040” has been used to designate both “Executing Sub-networks in the device(s) acc’ding to Received Configuration” in figure 10 and “device” in figure 14. Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim does not fall within at least one of the four categories of patent eligible subject matter because it is software per se.
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefore, subject to the conditions and requirements of this title. Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim is directed to “a computer program product comprising instructions”. When given the broadest reasonable interpretation “a computer program product comprising instructions” may be construed as software. Thus, the claims only recite software per se (descriptive material covered in MPEP 2106.01), which constitute as non-statutory subject matter.
Claims 1-7 and 10-12 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claims recite a mental process of observation, evaluation and judgment. This judicial exception is not integrated into a practical application, nor does it include additional elements that are sufficient to amount to significantly more than the judicial exception because the addition elements of the claims merely amounts to extra-solution activity in combination with generic computer hardware performing generic computer functions used to implement the abstract idea.
Claims 1 and 2
Step 1: The claim recites a method and an apparatus therefore, it falls into the statutory categories.
Step 2A Prong 1: The claim recites, inter alia:
determining availability and capabilities of at least one device; (This is a mental process of observation, evaluation and judgment wherein a user determines what devices are available and their capabilities, such as amount of memory and processing power.)
determining whether a modification is available in the at least one device or in capabilities of the at least one device, and if a modification is available, determining an updated or convolutional neural network configuration. (This is a mental process of observation, evaluation and judgment wherein a user looks at devices and determines if they capabilities and/or availabilities have changed. For example, a user determines a new device is available, so the user determines its capabilities and matches it a CNN configuration. In another case a user determines if the capabilities a device have changed, such as memory available decreasing, then matching it a CNN configuration with that requires less memory, which is consider an update.)
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Aside from the limitations above, the claim recites:
Using a convolutional neural network; at least one device; a controller; a processor (claim 2) – (All of the above element of the claims amount to using generic computer hardware to execute that abstract idea, see MPEP 2106.05(f).)
sending an updated or convolutional neural network configuration; sending an input image of parallel subnetworks to the at least one device; receiving output by the at one device to a controller to execute fusion of a processed image; (This amount to receiving data, which is data collection, and as such extra-solution activity, see MPEP 2106.05(g).)
executing subnetworks to process the input image in according to a received configuration; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level application of model to process data).
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer hardware performing generic functions that are implemented to perform the disclosed abstract idea above.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. The additional elements are:
Using a convolutional neural network; at least one device; a controller; a processor (claim 2) – (All of the above element of the claims amount to using generic computer hardware to execute that abstract idea, see MPEP 2106.05(f).)
executing subnetworks to process the input image in according to a received configuration; (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: high level application of model to process data).
sending an updated or convolutional neural network configuration; sending an input image of parallel subnetworks to the at least one device; receiving output by the at one device to a controller to execute fusion of a processed image; (This amount to receiving data, which is data collection, and as such extra-solution activity, see MPEP 2106.05(g). It is also well-understood, routine and conventional, see MPEP 2106.06(d)(II)(i) wherein it cites “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016)”.)
Claims 3 and 10
Step 2A Prong 1: The claim recites, inter alia:
Claims 3 and 10 inherit the abstract idea of claims 1 and 2.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Aside from the limitations above, the claim recites:
Uploading a neural network model to at least one device; (This amounts to receiving data, which is data collection, and as such extra-solution activity, see MPEP 2106.05(g). The device is also generic computer hardware used to implement the abstract idea, see MPEP 2106.05(f).)
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as they are mere insignificant extra solution activity in combination of generic computer hardware performing generic functions that are implemented to perform the disclosed abstract idea above.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. There are no additional limitations.
Uploading a neural network model to at least one device; (This amounts to receiving data, which is data collection, and as such extra-solution activity, see MPEP 2106.05(g). It is also well-understood, routine and conventional, see MPEP 2106.06(d)(II)(i) wherein it cites “The courts have recognized the following computer functions as well‐understood, routine, and conventional functions when they are claimed in a merely generic manner (e.g., at a high level of generality) or as insignificant extra-solution activity. i. Receiving or transmitting data over a network, e.g., using the Internet to gather data, Symantec, 838 F.3d at 1321, 120 USPQ2d at 1362 (utilizing an intermediary computer to forward information); TLI Communications LLC v. AV Auto. LLC, 823 F.3d 607, 610, 118 USPQ2d 1744, 1745 (Fed. Cir. 2016)”. The device is also generic computer hardware used to implement the abstract idea, see MPEP 2106.05(f).)
The additional elements as disclosed above in combination of the abstract idea are not sufficient to amount to significantly more than the judicial exception as they are mere insignificant extra solution activity that is well-understood, routine and conventional in combination of generic computer functions being implemented with generic computer elements to perform the disclosed abstract idea above.
Claims 4-5, 7 and 11-12
Step 2A Prong 1: The claim recites, inter alia:
Claims 4-5, 7 and 11-12 inherits the abstract idea to claims 1 and 2.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Aside from the limitations above, the claim recites:
Implemented for video encoding; implemented for video decoding; for playback using a processor. (The above limitations amount to linking the abstract idea to a technological field (neural networks with encoders/decoders), see MPEP 2106.05(h). The use of the processor for video playback is using generic computer hardware performing generic computer functions, see MPEP 2106.05(f).)
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as it mere links the abstract idea to a particular technological field in combination with generic computer hardware performing generic computer functions to implement the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. There are no additional limitations.
Implemented for video encoding; implemented for video decoding; for playback using a processor. (The above limitations amount to linking the abstract idea to a technological field (neural networks with encoders/decoders), see MPEP 2106.05(h). The use of the processor for video playback is using generic computer hardware performing generic computer functions, see MPEP 2106.05(f).)
The additional elements as disclosed above in combination of the abstract idea are not sufficient to amount to significantly more than the judicial exception as they merely link the abstract idea to a particular technological field in combination with generic computer hardware performing generic computer functions to implement the abstract idea.
Claims 6
Step 2A Prong 1: The claim recites, inter alia:
Claims 6 inherits the abstract idea of 2.
Step 2A Prong 2: This judicial exception is not integrated into a practical application. Aside from the limitations above, the claim recites:
an apparatus according to claim 2; and at least one of (i) an antenna configured to receive a signal, the signal including the coding unit, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes the coding unit, and (iii) a display configured to display an output representative of a coding unit. (All of the above claim limitations amount to using generic computer hardware performing generic computer functions to implement the abstract idea, see MPEP 2106.05(f).)
The additional elements as disclosed above alone or in combination do not integrate the judicial exception into practical application as it is generic computer hardware performing generic computer functions to implement the abstract idea.
Step 2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception. There are no additional limitations.
an apparatus according to claim 2; and at least one of (i) an antenna configured to receive a signal, the signal including the coding unit, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes the coding unit, and (iii) a display configured to display an output representative of a coding unit. (All of the above claim limitations amount to using generic computer hardware performing generic computer functions to implement the abstract idea, see MPEP 2106.05(f).)
The additional elements as disclosed above in combination of the abstract idea are not sufficient to amount to significantly more than the judicial exception as they are mere generic computer hardware performing generic computer functions to implement the abstract idea.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-7 and 9-12 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention. Claims 1 and 2 cites “sending an input image of parallel sub-networks to the at least one device.”, however a review of the specification does not mention “an input image of parallel sub-networks”, nor what exactly is an image of parallel subnetworks. Additionally claim 6 says “the coding unit” but there is no mention of a coding unit in the specification. As claims 1, 2 and 6 are rejected under 35 USC 112(a) for failing to comply with written description, then so does claims 3-5, 7, and 9-12 are rejected as being dependent on claims 1 and 2.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1 and 2 are rejected using 35 USC 112(b) for be indefinite. Claims 1 and 2 cite a “device in a convolutional neural network (CNN)”, however it is not clear to the examiner from a review of the specification how a device is in CNN, as such the examiner finds the claim to be unclear and indefinite. In the interest of compact prosecution, the examiner interprets the claim to mean determining availability and capabilities of at least one device for use in executing a convolutional neural network as the claim teaches distributing portion of a CNN to edge devices. Additionally claims 1 and 2 cites “sending an input image of parallel sub-networks to the at least one device.”, however a review of the specification does not mention “an input image of parallel sub-networks”, nor what exactly is an image of parallel subnetworks, as such it is not clear to the examiner what is meant by this and what is required to meet the claim limitation, such the claim is found in be indefinite. In the interest of compact prosecution, the examiner interprets this limitation to mean “sending an input image to at least one device containing or implementing part of the parallel subnetwork”.
In regards to claim 4 it is rejected under 35 USC 112(b) for being indefinite. Claim 4 cites “the method of claim” but fails to disclose what claim that claims 4 is claiming dependency to, as such it is found to be unclear and indefinite. In the interest of compact prosecution, the examiner interprets claim 4 to be dependent on claim 1.
Claim 6 is rejected under 35 USC 112(b) for being indefinite. Claim 6 recites the limitation "“the coding unit” in line 4 of claim 6. There is insufficient antecedent basis for this limitation in the claim. Additionally, it is not clear what is meant by a “coding unit” as it is not disclosed or mentioned in the specification. As such it is unclear to the examiner what is meant by a coding unit and what is output representing a coding unit.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 6, and 9-10 are rejected under 35 U.S.C. 103 as being unpatentable over Bhardwaj et al. (“Memory- and Communication-Aware Model Compression for Distributed Deep Learning Inference on IoT” – hereinafter Bhardwaj) in further in view of Almeida et al. (US 20220083386 A1 – hereinafter Almeida) .
In regards to claim 1, Bhardwaj discloses a method, comprising:
determining capabilities of at least one device in a convolutional neural network; (Bhardwaj page 7 section 4.1, and equation 2 teaches selecting student sizes based on memory and computation budget and allocating workloads to devices wherein it teaches student constraints on memory and FLOPS and partitioning based on them.)
sending an input image of parallel sub-networks to the at least one device; (Bhardwaj page 16 section 6.1 teaches sending an input image to devices wherein it cite “In addition, we use an x86 machine (Core i7), further referred as server, to send the images to the devices, concatenate the results, and apply the fc layer.”. Also see page 10-11 section 4.3.2. teaches student networks on individual devices execute their respective subnetworks. It teaches each subnetwork have identical initial layers and different layers after that.)
executing sub-networks to process the input image in the at least one device according to a received configuration; and (Bhardwaj page 16 section 6.1 teaches sending an input image to devices wherein it cite “In addition, we use an x86 machine (Core i7), further referred as server, to send the images to the devices, concatenate the results, and apply the fc layer.”.)
receiving output by the at least one device to a controller to execute fusion of a processed image. (Bhardwaj page 16 section 6.1 teaches a sever receiving and concatenating results and applying a fully connected layer wherein it cites “In addition, we use an x86 machine (Core i7), further referred as server, to send the images to the devices, concatenate the results, and apply the fc layer.”. Also see page 19 section 6.2.3 that teaches “…receiving and concatenating results from all students.” These teach fusion of output from students processing the image.)
However, Bhardwaj does not explicitly disclose determining availabilities and capabilities of at least one device in a convolutional neural network; and determining whether a modification is available in the at least one device or in capabilities of the at least one device, and if a modification is available, determining and sending an updated or convolutional neural network configuration.
Almeida determining availabilities and capabilities of at least one device in a convolutional neural network; and (Almeida para. [0058-0061] teaches identifying available computing resources and their loads, selecting suitable resources and determining partition sizes based on computing capability, processor and load. and determining whether a modification is available in the at least one device or in capabilities of the at least one device, and if a modification is available, determining and sending an updated or convolutional neural network configuration. Also, para. [0075] teaches device status and load information for dynamic repartitioning and reassignment.) determining whether a modification is available in the at least one device or in capabilities of the at least one device, and if a modification is available, determining and sending an updated or convolutional neural network configuration. (Almeida para. [0063] teaches receiving a message that a resource/device can no longer complete execution because its load has changed, and dynamically changing distribution based on the new information. Also, para. [0075] teaches device-status and load information for dynamic repartitioning or reassignment, as well as communicating data for each partition directly or through the hub.)
It would have been obvious to one of ordinary skill in the art before the earliest effective filing date of the claimed invention to modify the teachings of the Bhardwaj with Almeida’s device status monitoring and dynamic partition assignment to deliver revised or update assignments using Bhardwaj model deployment as both reference deal with using neural networks. It provides the benefit of creating a more efficient network by allowing assignment based on availability and capabilities of devices thus maximizing processing of each device.
In regards to claim 2, it is the apparatus embodiment of claim 2 with similar limitations to that of claim and thus is rejected using the same reasoning as that found in claim 1. The only difference between is that 2 cites the use of a processor, however that is disclosed in both Bhardwaj (Page 16 section 6.1 teaches a quad core processor) and Almeida (fig. 2 element 108 teaches a processor.) references.
In regards to claim 3, Bhardwaj in view of Almeida discloses the method of claim 1, further comprising uploading a neural network model to at least one device. (Bhardwaj page 16 section 6.1 teaches sending a neural network to student devices, this would be uploading a neural network to at least one device.)
In regards to claim 6, it is a device embodiment of claim 2 with similar limitations and thus is rejected using the same reason as that found in claim 2. The only difference is that claim 6 cites “a device comprising: an apparatus according to claim 2; and at least one of (i) an antenna configured to receive a signal, the signal including the coding unit, (ii) a band limiter configured to limit the received signal to a band of frequencies that includes the coding unit, and (iii) a display configured to display an output representative of a coding unit.” Which is disclosed by Almeida para. [0070] which teaches a display screen to show output and see figure 5 wherein devices send information back and forth between each other and a cloud, thus they much have an antenna to receiving signals with data.
In regards to claim 9, Bhardwaj in view of Almeida disclose a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the method of claim 3. (Almeida para. [0032-0036] teaches a computer program product with instructions carried out by a computer.)
In regards to claim 10, Bhardwaj in view of Almeida disclose the apparatus of claim 2, further comprising uploading a neural network model to at least one device. (Bhardwaj page 16 section 6.1 teaches sending a neural network to student devices, this would be uploading a neural network to at least one device.)
Claims 4-5, 7, and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Bhardwaj et al. (“Memory- and Communication-Aware Model Compression for Distributed Deep Learning Inference on IoT” – hereinafter Bhardwaj) in further of Almeida et al. (US 20220083386 A1 – hereinafter Almeida) above, and further in view of Dumas et al. (WO 2021089494 A1 – hereinafter Dumas).
In regards to claim 4 and 5, Bhardwaj in view of Almeida discloses the method of claim 1, but fails to explicitly disclose implemented for video encoding and decoding.
Dumas discloses implementing a neural network for video encoding and decoding. (Dumas page 1 teaches “The method comprises steps for the aforementioned training of a set of neural networks for intra prediction and further comprises performing encoding or decoding of a video block using the generated set of neural networks.”)
It would have been obvious to one of ordinary skill in the art before the earliest effective filing date of the claimed invention to modify the teachings of the Bhardwaj in view of Almeida with the teachings of Dumas in order to allow for using a neural network for video encoding/decoding as all the reference deal using a neural network and Almedia and Dumas deal with video usage. It provides the benefit of creating a more robust system by allowing the neural network to use videos, thus expanding the capabilities of the system beyond just images.
In regards to claim 7 , Bhardwaj in view of Almeida discloses a non-transitory computer readable medium containing data content generated according to the method of claim 1, (Almedia para. [0031, 0033 and 0035-0036] teaches a computer readable medium and non-transitory data with code or instructions.), however it does not explicitly disclose for playback using a processor.
Dumas discloses using a processor for playback. (Dumas claim 13 cites “A non-transitory computer readable medium containing data content generated according to the method of any one of claims 1 through 3 and 7 to 11, or by the apparatus of any one of claims 2, and 5 through 11, for playback using a processor.”)
It would have been obvious to one of ordinary skill in the art before the earliest effective filing date of the claimed invention to modify the teachings of the Bhardwaj in view of Almeida with the teachings of Dumas in order to allow for using a processor for video playback as all the reference deal using a neural network and Almedia deal with video usage. It provides the benefit of allowing a user to check the output of the neural networks to makes sure video is correct and what is required.
In regards to claim 11 and 12, it is the apparatus embodiment of claims 4 and 5 with similar limitations and thus rejected using the same reasoning found in claims 4 and 5.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PAULINHO E SMITH whose telephone number is (571)270-1358. The examiner can normally be reached Mon-Fri. 10AM-6PM CST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Abdullah Kawsar can be reached at 571-270-3169. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PAULINHO E SMITH/Primary Examiner, Art Unit 2127
Prosecution Insights