Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The amendments provided 03/12/2026 have been entered and considered. Claims 1, 15, and 17 have been amended, claim 3 has been cancelled.
Response to Arguments
Prior art rejections
On pages 12-13 of the remarks (03/12/26) applicant contends that the cited combination of using Liu is improper based on no motivation to combine with Chaung/Navarrete Michelini.
The examiner respectfully disagrees. The examiner recognizes that obviousness may be established by combining or modifying the teachings of the prior art to produce the claimed invention where there is some teaching, suggestion, or motivation to do so found either in the references themselves or in the knowledge generally available to one of ordinary skill in the art. See In re Fine, 837 F.2d 1071, 5 USPQ2d 1596 (Fed. Cir. 1988), In re Jones, 958 F.2d 347, 21 USPQ2d 1941 (Fed. Cir. 1992), and KSR International Co. v. Teleflex, Inc., 550 U.S. 398, 82 USPQ2d 1385 (2007). In this case, Chaung shows that the queuing of tasks is an operation performed by the system to execute super-resolution processing (see Fig. 10). As such, operations that improve this aspect have a reason for combination (as was provided in the Non-final rejection of 01/16/2026). As such, the data consistency operation performed by Liu is applicable, by virtue of its impact to avoid incorrectly queued instructions (such incorrect read/write operation queuing is avoided by the read/write consistency operations performed per Liu [0023]).
Further, regarding applicant's argument that Liu is nonanalogous art, it has been held that a prior art reference must either be in the field of the inventor’s endeavor or, if not, then be reasonably pertinent to the particular problem with which the inventor was concerned, in order to be relied upon as a basis for rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992). As stated prior, it is directly relevant by virtue of Chaung incorporating queuing operations. Thus, the combination is reasonably pertinent to the operations of Chaung. Additionally, it is not understood how the incorporation of Liu would destroy the intended operation of Chaung/Navarrete Michelini. Chaung uses queue operations to read and execute program instructions from storage, thus providing a reason for Liu’s application while aiding in the already performed reading operations of Chaung. Also, the relied upon combination is specifically for the sake of data consistency checking for the denoted benefit mentioned on page 16 of the non-final rejection (01/16/2026), which is with respect to reading/execution operations between modules (Navarrete Michelini and Chaung having a plurality of modules in communication with one another, as shown in the reasons for rejection below).
On pages 15-18 of the remarks (03/12/26) applicant contends that the cited combination of prior art does not address the amended claim limitations.
The examiner respectfully disagrees, specifically (as is further described in the 103 rejection below) Chaung in combination with Navarrete Michelini provided the required architecture of the amended claim limitations.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitations uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
Claim 1 lines 16-18 “…a first data dependency determination unit configured to ensure consistency in reading data from and writing data to a first storage unit associated with the primary operation module…”
Claim 1 lines 18-21 “…and each of the plurality of secondary operation modules includes a second data dependency determination unit configured to ensure consistency in reading data from and writing data to a second storage unit associated with the respective secondary operation module…”
Claim 15 lines 14-16“…a first data dependency determination unit configured to ensure consistency in reading data from and writing data to a first storage unit associated with the primary operation module…”
Claim 15 lines 16-18“…and each of the plurality of secondary operation modules includes a second data dependency determination unit configured to ensure consistency in reading data from and writing data to a second storage unit associated with the respective secondary operation module…”
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have these limitations interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8, 9, 11, and 15-18 are rejected under 35 U.S.C. 103 as being unpatentable over Chaung et al.( US Publication 20220058772 A1; hereinafter “Chaung”) in view of Navarrete Michelini et al. ( US Publication 20180315165 A1; hereinafter “Navarrete Michelini”) in further view of Liu et al. (WO publication 2017124642; hereinafter “Liu” (Liu corresponds to EP publication 3407265, which is relied upon as an English translation of the WO publication, so the citations below reference the corresponding EP publication)).
In re to claim 1, Chuang teaches wherein: an information processing method (image processing method; title and abstract, discloses a the processing of image data, understood to be information processing) applied to a computation device (mobile phone; Fig. 2 (200). See also [0063] which indicates that the processes applicable for Fig. 1 are also applicable for Fig. 2), wherein the computation device includes a communication unit (memory and its connection architecture; Fig. 2 (230), which per Fig. 2, is in communication with the processor, via the connections shown in Fig. 2), a register unit ([0147] discloses the use of a register in order to store operation instructions of a software module. It is understood that the register, when used as a form of memory for the system, is the register unit), a controller unit (application layer; Fig. 2 shows an application layer that has application programs stored within it that is in further communication with the image processing module in order to execute the processes of the system (as is further corroborated by [0061] and [0070])), and an operation unit, the information processing method comprises: controlling, by the computation device, the communication unit to obtain a first image to be processed ([0064] discloses that the processor controls the functions of the phone, which per Fig. 2, it comprises. Further, [0064] lines 5-9 discloses that the processor performs tasks using the memory, thus disclosing what is understood to be controlling the memory. See also [0061] lines 6-8, “The image processing module 122 receives an image transmitted by the first application program 111 and is configured to perform image optimization processing on the image.” Which is understood to disclose obtainment of the first image (the to-be-displayed image when it requires super-resolution processing, as it is described in [0062]). Said image is received from the first application program. Additionally, per [0064] lines 5-9, the applications are stored in the memory. Thus, the first image (correspondent to the claims) is obtained from the memory, as the first application program comprises said memory), wherein the first image has a resolution of a first-level size ([0062] lines 1-4 discloses determination of the resolution of the first image (correspondent to the claims) in relation to a threshold (the value less than the threshold being the first-level size));
controlling, by the computation device, the controller unit to fetch an operation instruction from the register unit, and sending, by the computation device, the operation instruction to the operation unit ([0147] discloses the use of a register in order to store operation instructions of a software module. Further, ([0062] discloses the performance of the super resolution processing by the image processing module.[0064] additionally states that said processor performs execution of programs and data for the device of Fig. 2 (200). See also [0061] and Fig. 2, which show the use of an application layer that contains application programs that are in communication with the image processing module. It is understood that the execution of the operation instructions due to communication with the application layer is the performance of a fetch operation with the controller unit);
controlling, by the computation device, the operation unit to obtain and execute an operation instruction to perform resolution optimization ([0062] discloses the performance of the super resolution processing by the image processing module. The super resolution processing being understood to be the operation instruction that performs resolution optimization. Per [0062] this is done on the first image (correspondent to the claims). Further, this image processing module’s performed actions are understood to be performed by the processor of Fig. 2 (220). [0064] states that said processor performs execution of programs and data for the device of Fig. 2 (200). See also [0063], which state it to be able to perform the processes of Fig. 1) to obtain a second image, wherein the second image has a resolution of a second-level size, wherein the first-level size is smaller than the second-level size (image obtained after super resolution processing; [0062] discloses the action of performing super resolution processing to produce an image with increased resolution. The increased resolution is understood as the second-level size), and the operation instruction is a preset instruction for optimizing an image resolution ([0062] discloses that the system performs super resolution processing. Thus, as this is a programmatic action ([0064] describes functions of the phone to be executed software programs through the processor), it is understood that the operation instruction (correspondent to the claims) is a preset instruction. As for the processor to execute a function, it is a software program or data stored in the memory (see [0064] lines 23-27)).
As well as teaching calling the operation instruction to perform resolution optimization on the first image includes: controlling, by the computation device ([0062] discloses the performance of the super resolution processing by the image processing module (which subsequently produces the second image correspondent to the claims). Further, per Fig. 2, it is a part of the computation device (correspondent to the claims). Thus, this processing is performed on the first image (as per the [0062]))
Chaung does not explicitly teach wherein: controlling, by the computation device, the operation unit to call the operation instruction to perform feature extraction on the first image to obtain a feature image;
controlling, by the computation device, the operation unit to pre-process the feature image to obtain a second image, wherein the pre-processing is an operation, nor the operation unit includes: a primary operation module configured to control data flow and aggregate computation results, and a plurality of secondary operation modules, wherein each of the plurality of secondary operation modules is configured as a parallel convolution computation engine, wherein the primary operation module is interconnected with the plurality of secondary operation modules by an interconnection module, and when the operation instruction is a convolution operation instruction, performed by the system to change resolution, the plurality of secondary operation modules to implement a convolution operation of input data and a convolution kernel in a convolutional neural network algorithm, wherein the input data is the first image and the convolutional neural network algorithm corresponds to the convolution operation instruction, controlling, by the computation device, the interconnection module to implement data transfer between the primary operation module and the plurality of secondary operation modules, before a forward operation of a neural network fully connected layer starts, transferring, by the primary operation module, the input data to each of the plurality of secondary operation modules through the interconnection module, such that the plurality of secondary operation modules receive the same input data for parallel computation, and after the computation of the plurality of secondary operation modules is completed, splicing, by the interconnection module, a plurality of output scalars computed in parallel by the respective secondary operation modules stage by stage to obtain an intermediate vector, and sending the intermediate vector back to the primary operation module, and controlling, by the computation device, the primary operation module to splice intermediate vectors corresponding to all input data into an intermediate result for performing subsequent operations, nor does Chaung explicitly teach the plurality of secondary modules use the same input data and respective convolution kernels to compute respective output scalars in parallel.
However, in a related field of endeavor, Navarrete Michelini teaches wherein: controlling, by the computation device, the operation unit to call the operation instruction to perform feature extraction on the first image to obtain a feature image ([0021]discloses the conversion of an input image into a plurality of feature images, which is understood to disclose a feature extraction operation in order to generate said feature images);
controlling, by the computation device, the operation unit to pre-process the feature image to obtain a second image, wherein the pre-processing is an operation([0022] discloses the integration of the feature images into a multiplexer, which is understood as a pre-processing operation), as well as the operation unit includes: a primary operation module (collection of bias adding portions of convolutional layers, and the initial data input of the input signal image; the abstract and Fig. 4 indicate that the system is a convolutional neural network. Thus, the system adds biasing (see also [0040], which discloses the inclusion of bias). Further, it is understood that this portion, in combination with the initial data entry point that feeds into the first convolutional neutral network (see Fig. 2E, and [0033] which describes that the system initially receives a first image signal as input data) are the primary operation module) configured to control data flow and aggregate computation results ([0040] indicates that the primary operation module controls data flow by virtue of processing data and directing it to further portions of the system (see also Fig. 4 which shows the flow of data in the system architecture). Additionally, as the system outputs the results of biasing to particular further portions of the system (see [0040]), it is understood to aggregate computation results (being the biased output)), and a plurality of secondary operation modules (portion of filter circuits that perform convolution for the neutral network; abstract and Fig. 4 indicate use of convolutional layers within a neutral network (as further described in [0039]). Each layer is understood to perform a convolutional operation, as can be seen in Fig. 4. Additionally, [0033] discloses that the system utilizes its convolutional layers via filters to generate output feature images), wherein each of the plurality of secondary operation modules is configured as a parallel convolution computation engine ([0033] discloses that the filters utilize the same input data to generate the plurality of feature images. Further, as this is done across a plurality of filters with each producing their own image, it is additionally understood to be parallel computation. Thus, the collection of secondary operation modules (correspondent to the claims) is understood to constitute a parallel convolution computation engine), wherein the primary operation module is interconnected with the plurality of secondary operation modules by an interconnection module (combination of circuit connections that connect the portions within a neural network; Fig. 4 discloses the connection of layers within a neutral network, which, as per Fig. 4 and [0040] includes connection to the portion of the convolutional layer that introduces bias), and when the operation instruction is a convolution operation instruction, performed by the system to change resolution (the title and abstract disclose that the operations of the system are for the alteration of image scaling, and thus image resolution, by upscaling), the plurality of secondary operation modules to implement a convolution operation of input data ([0033] discloses that the system uses an image of an input signal and processes it to output feature images. See also Fig. 4, which shows an example of convolutional layers within a neutral network, as described in [0039]. Each layer is understood to perform a convolutional operation, as can be seen in Fig. 4. Thus, it is understood that the secondary operation modules (correspondent to the claims) perform a convolution operation (understood as the action of convolution)) and a convolution kernel ([0040] lines 6-7 discloses the equivalence of filters within convolutional layers to that of a kernel (and as these kernels are a part of the convolutional process, as suggested by the generation of feature images described in [0033], they are understood as convolution kernels)) in a convolutional neural network algorithm (the abstract and Fig. 4 indicate the system to be a convolutional neutral network. As such it is understood to teach the convolution kernel within its convolutional neural network algorithm (understood to be the convolutional neural network operations undergone in Fig. 2E)), wherein the input data is the first image and the convolutional neural network algorithm corresponds to the convolution operation instruction (the abstract discloses that the system takes an image as input, understood to disclose the first image as input data. See also [0033], which discloses use of the input data to generate feature images. Further, as the convolutional neural network algorithm (correspondent to the claims) comprises the various operations of the convolutional neutral network systems of Fig. 2E, it is understood to thus correspond to the convolution operation instruction (correspondent to the claims)), controlling, by the computation device, the interconnection module to implement data transfer (Fig. 4 discloses the connection of layers within a neutral network. Further, as is disclosed in [0039], layers act upon an image as an input, as such, the layers must receive this image. Thus, disclosing data transfer) between the primary operation module and the plurality of secondary operation modules (Fig. 2E shows the convolutional neutral network and it shows contiguous connection between portions of the system from input to output, showing data transfer through the use of the interconnection module (correspondent to the claims). Additionally, as shown in both Fig. 4 and [0040], there is connection to the bias adding portion of the convolutional layer. Further, as convolution occurs in another portion of the layer, it is understood that the connection enables the transfer of data between the primary and secondary operation modules (each correspondent to the claims respectively)), before a forward operation of a neural network fully connected layer starts (Fig. 4 shows a point prior to a final convolutional layer. Thus, prior to a forward operation (being the forward propagation through a neural network) within a fully connected neural network layer (a convolutional layer connected to another layers of the system and an output layer, as is shown by Fig. 4) data is transferred to said fully connected layer. See also Fig. 2E which shows a plurality of neural networks, with each transfer of data occurring before the last network displaying transfer prior to a forward operation of the last neural network fully connected layer’s start), transferring, by the primary operation module, the input data to each of the plurality of secondary operation modules through the interconnection module ([0033] discloses that the first convolutional neutral network receives the input data to its plurality of filter. See also Fig. 2E, which shows that there is a point that feeds into the first convolutional neutral network circuit, thus displaying transfer of the input data to the secondary operation module by the primary operation module (each correspondent to the claims respectively)), such that the plurality of secondary operation modules receive the same input data for parallel computation ([0033] discloses that the filters utilize the same input data to generate the plurality of feature images. Further, as this is done across a plurality of filters with each producing their own image, it is additionally understood to be parallel computation. Thus, the collection of secondary operation modules receives the same input for parallel computation), and after the computation of the plurality of secondary operation modules is completed, splicing, by the interconnection module (Fig. 4 shows that convolutional layers reduce the number of images they produce as they propagate through the neutral network with the utilization of the interconnection module (correspondent to the claims). Further, Fig. 4 and [0033-0034] disclose the combination of outputs to form a consolidated multiplexer output (understood to be splicing)), a plurality of output scalars (features of a feature image; the abstract discloses the operation of generating feature images. These image are comprised of features, which are understood to be scalar values. See also [0033], which discloses the generation of feature images by filters) computed in parallel by the respective secondary operation modules stage by stage to obtain an intermediate vector (Fig. 3 shows the splicing (correspondent to the claims) of data. Further, as this is done between two convolutional layers (per [0034]), it is understood to be representative of being done between respective secondary operation modules (correspondent to the claims). This is stage by stage due to being through the neutral network’s forward propagation. Each network layer and multiplexer is understood to be a stage, thus, disclosing a stage by stage action. This is done to generate upscaled images as a product of the propagation (as is disclosed in [0034]). These resultant images generated prior to the finalized output of a given neutral network layer and multiplexer pairing are understood to be the intermediate vectors), and sending the intermediate vector back to the primary operation module (Fig.4 shows the interconnection of different portions of the system, including bias adding portions of the neural network. It is further understood that by its data being subsequently sent to a bias adding portion in the next layer (see Fig. 4), that it is sent back to the primary operation module (correspondent to the claims). Thus, disclosing the intermediate vector being sent back to the primary operation module (each correspondent to the claims, respectively)), and controlling, by the computation device, the primary operation module to splice intermediate vectors corresponding to all input data into an intermediate result for performing subsequent operations (Fig. 4 shows the generation of an intermediate vector (correspondent to the claims). Subsequently, when said vector is processed into a final processing point prior to a second convolutional neutral network (as shown in Fig. 2E), it is understood to be an intermediate result. Additionally, as this is a splice (correspondent to the claims) signal of the last level of the apparatus, it is understood to correspond to all input data, with subsequent operations being the operations of the second convolutional neutral network (see [0030])) As well as teaching the plurality of secondary modules use the same input data and respective convolution kernels to compute respective output scalars in parallel ([0033] discloses that the filters utilize the same input data to generate the plurality of feature images. Further, as this is done across a plurality of filters with each producing their own image, it is additionally understood to be the parallel computation of scalars by respective convolution kernels, as the secondary modules comprise said kernels (each correspondent to the claims respectively)).
Navarrete Michelini, like Chuang, teaches the alteration of an input image’s resolution to that of a higher resolution. This being a preset design element of their respective systems.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Chuang, to implement a convolutional neural network that includes the various elements required to perform convolutional operations, as taught by Navarrete Michelini, to arrive at the claimed invention discussed above. The motivation for the proposed modification would allow for the system to integrate the adaptability of a convolutional neural network to better address varied images of different image data. Allowing for the upscaling of images to better fit an individual image, as is done in Navarrete Michelini Fig. 2E’s upscaling apparatus system. Further, as is the case for Navarrete Michelini Fig. 4, doing so in parallel reduces the amount of time for the execution of the system as opposed to doing the convolutions that are not in parallel.
Chaung, in view of Navarrette Michelini further teaches wherein: the pre-processing is an operation preset by a user side or a terminal side (Chaung [0061] discloses the operation of programs from the application layer of a user device (see [0067] line 6 discloses that the system that is comprised of the application layer is a user’s mobile phone). Additionally, as these programs are called to perform operations of the system, they are understood to be preset on the user side).
Chaung, in view of Navarrete Michelini, does not explicitly teach wherein: the primary operation module includes a first data dependency determination unit configured to ensure consistency in reading data from and writing data to a first storage unit associated with the primary operation module, and each of the plurality of secondary operation modules includes a second data dependency determination unit configured to ensure consistency in reading data from and writing data to a second storage unit associated with the respective secondary operation module of the plurality of secondary operation modules, nor wherein the first data dependency determination unit sends data read from the first storage unit to the plurality of secondary operation modules through the interconnection circuit, nor wherein each second data dependency determination unit is configured to ensure there is no consistency conflict between reading and writing of the data.
However, in a related field of endeavor, Liu teaches wherein: the primary operation module includes a first data dependency determination unit configured to ensure consistency in reading data from and writing data to a first storage unit associated with the primary operation module ([0019] discloses the use of a data dependency relationship unit (unit 52), understood to be the first data dependency determination unit, for a master computation module (understood as the primary operation module). Further, per [0020], it performs read/write operations for data consistency to a cache unit (cache unit 53), understood as the first storage unit associated with the primary operation module by virtue of its connection to unit 52), and each of the plurality of secondary operation modules includes a second data dependency determination unit configured to ensure consistency in reading data from and writing data to a second storage unit associated with the respective secondary operation module of the plurality of secondary operation modules ([0021] shows the use of a data dependency relationship unit (unit 62), understood to be the second data dependency determination units, for a plurality of slave computation modules (understood as the secondary operation modules). Further, see Fig. 4 which is representative of the structure of the plurality of slave computation modules. Further, per [0021], it performs read/write operations for data consistency to a cache unit (cache unit 63), understood as the second storage unit associated with the secondary operation modules by virtue of their connection to unit 62. Thus, each secondary module is understood to have a respective unit 62 and unit 63), as well as wherein the first data dependency determination unit sends data read from the first storage unit to the plurality of secondary operation modules through the interconnection circuit ([0020] lines 5-9 discloses use of the first data dependency determination unit (correspondent to the claims) to that of the secondary operation modules (correspondent to the claims). It is understood that the connections that enable information transfer between units constitutes the interconnection circuit), as well as wherein each second data dependency determination unit is configured to ensure there is no consistency conflict between reading and writing of the data ([0021] discloses that the second data dependency determination units (correspondent to the claims) perform read/write operations for data consistency).
Liu, like Chuang, teaches a neural network system that processes data for the sake of image processing.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Chuang, in view of Navarrete Michelini, to implement consistency determination operations, as taught by Liu, to arrive at the claimed invention discussed above. The motivation for the proposed modification would be to reduce the likelihood that the output processed image data of the super resolution operation of the system is incorrectly performed or negatively impacted by incorrectly queued instructions (such incorrect read/write operation queuing is avoided by the read/write consistency operations performed per Liu [0023]).
In re to claim 2 [dependent on claim 1], Chuang teaches wherein: the controlling, by the computation device, the communication unit ([0064] lines 23-27 discloses the processor in order to execute functions, utilizes software programs or data stored in the memory) to obtain an original image to be processed, input by a user ([0077]-[0078] discloses user interaction with the phone, choosing an image for the system to perform super resolution on. Thus disclosing user input of said image, understood to be an original image, which is further understood to be the image chosen by the user for processing. Additionally, as per [0077], the original image (correspondent to the claims) is sent by the first application program, which comprises the memory (see [0064] lines 5-9), and thus discloses that said image is obtained from the memory), wherein the original image has a resolution of the first-level size ([0077]-[0078] discloses that the original image, correspondent to the claims, has super-resolution performed on it. Thus, it is below a threshold, causing it to have the first-level size (correspondent to the claims)), and controlling, by the computation device, the operation unit to pre-process the original image to obtain the first image to be processed ([0078] discloses that the original image is compared to a threshold, this comparison being understood to be pre-processing. Further, by performing this comparison, the image is determined to have the first-level size (correspondent to the claims), making it the first image to be processed. As super-resolution processing will be performed as a result of having this first-level size (correspondent to the claims)), wherein the pre-processing is an operation preset by a user side or a terminal side ([0064] discloses functions of the phone to be executed software programs through the processor. Thus, the super-resolution is understood to be a preset instruction (the processor to execute a function, it is a software program or is data stored in the memory (as per [0064] lines 23-27)). Additionally, in order for super resolution to occur, the comparison to the threshold must first be performed, this being understood to be a part of performing super-resolution processing as suggested by [0062]. As this is a program on the phone, it is also understood to be terminal side. With the device the user is utilizing being understood to be the terminal).
In re to claim 4 [dependent on claim 1], Chaung does not explicitly teach wherein: the pre- processing includes one or more of the following processing: translation, scaling transformation, non-linear transformation, normalization, format conversion, data deduplication, processing of data exception, and data missing filling.
However, in a related field of endeavor, Navarrete Michelini teaches wherein: the pre-processing includes one or more of the following processing: translation, scaling transformation, non-linear transformation, normalization, format conversion, data deduplication, processing of data exception, and data missing filling (abstract, discloses that the multiplexer upscales the image. Which is understood to be scaling transformation, and thus pre-processing includes scaling-transformation. Further, as this is up-scaling that occurs prior to the generation of the final output, performed by the multiplexer, it is understood to be included in pre-processing).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective
filing date of the claimed invention to modify Chuang, scaling transformation, as taught by Navarrete Michelini, to arrive at the claimed invention discussed above. The motivation for the proposed modification would provide an explicit method of altering resolution for the image during processing within a neural network. Chaung teaches already to alter the resolution through super resolution, in Chaung [0062], allowing it to upscale an image for increased clarity. Providing a method to upscale the image when utilizing a neural network allows for the upscaling to be better fit per individual image. While also allowing preserving the goal of increased visual clarity by having a larger image for the user to view.
In re to claim 5 [dependent on claim 1], Chaung teaches wherein: the calling the operation instruction to perform functions ([0062] discloses the performance of the super resolution processing by the image processing module. Thus, disclosing in the call of the operation instruction (correspondent to the claims), which is a function of the system as it is a task carried out by the system. Further, as per [0064], it is disclosed that the processor performs execution of programs and data for the device of Fig. 2 (200) from information stored in memory. Which further discloses that the operation instruction is one that is called, as it is accessed from the memory) based on an operation instruction set ([0064] discloses that the processor performs execution of programs and data for the device of Fig. 2 (200) from information stored in memory. Which further discloses that the operation instruction (correspondent to the claims) is one that is called, as it is accessed from the memory. Additionally, there is a program that performs user interaction (see [0072], which discloses that one of the application programs is software to enable user interfacing). Thus, as there are multiple programmatic instructions stored in the memory, such as super resolution processing and user interfacing, it is understood that an operation instruction set is disclosed by the multitude of programs stored in memory (see [0064] which states that the memory may be the storage unit of software programs within the system)) further teaching wherein the operation instruction set includes at least one operation instruction ([0064] discloses that the processor performs execution of programs and data for the device of Fig. 2 (200) from information stored in memory. [0062], disclosing the performance of the super resolution processing by the image processing module, and as a result, by the processor. Thus, as this super resolution is one of the executed programs of the system, and there are other programs, it is understood that the operation instruction is included in the operation instruction set (each correspondent to the claims respectively)), and an order of calling the operation instruction in the operation instruction set is customized by a user side or a terminal side ([0077]-[0078], discloses user interaction with the phone, choosing an image for the system to perform super resolution on. Thus, disclosing that the action of calling the operation instruction (correspondent to the claims) is customized by a user side action. With the customization being the user based instruction (through interaction) of what image to apply super resolution to).
Chaung does not explicitly teach wherein: to perform feature extraction on the first image to obtain a feature image includes: the operation unit to perform feature extraction on the first image nor does it teach activation of at least one thread to obtain a feature image
However, in a related field of endeavor, Navarrete Michelini teaches wherein: to perform feature extraction on the first image to obtain a feature image includes: the operation unit (entire neutral network system; Fig. 2E) to perform feature extraction on the first image (abstract lines 1-4 discloses feature extraction to obtain a feature image, as it discloses generation of a feature image from input images, with this generation being understood to be the feature extraction. See also [0028] and [0033], which discloses that the first convolutional neutral networks are what obtains the feature images. Additionally, while this is stated for Fig. 2C and 2D, Fig. 2E shares the cascading configuration of convolutional neutral networks and multiplexers, and as such performs feature extraction in the same way as the other figures (Fig. 2C and 2D). As the differentiating factor of a second convolutional neutral network (as is described in [0030]) is used to enhance output image quality. Which does not impact feature image generation. Thus, the performance of the feature extraction by the operation circuit (correspondent to the claims) is disclosed for Fig. 2E. Further, as is indicated by the abstract, this is performed on an input image, and thus on the first image (correspondent to the claims)) and it teaches activation of at least one thread to obtain a feature image (Fig. 4 displays the structural diagram of a convolutional neutral network (see also [0013] for this distinction). The network is comprised of multiple connecting lines, understood to be threads. Additionally, as per the abstract, [0028], and [0033], a convolutional neutral network obtains the feature image. Thus, disclosing at least one thread is in activation to obtain a feature image).
Navarrete Michelini, like Chuang, teaches the alteration of an input image’s resolution to that of a higher resolution. This being a preset design element of their respective systems.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective
filing date of the claimed invention to modify Chuang, to perform feature extraction , as
taught by Navarrete Michelini, to arrive at the claimed invention discussed above. The motivation for the proposed modification would have been to enable for the automatic processing of data when generating an upscaled image, which also allows for the system to better address images individually, as with a convolutional neural network, the process of altering the resolution will change in respect to the image. Allowing the process to be fitted to said image.
In re to claim 6 [dependent on claim 1], Chaung teaches wherein: the computation device further includes a data access circuit (connected processor and storage medium; Fig. 2) and a storage medium ([0147] discloses the various forms of memory utilized by the system to store the software module. Which, given that it is what is executed by the processor a system memory, it is indicative of the software stored in the memory, as per [0064]. Thus, as this is describing what the software storage unit may be, it is understood to further disclose there being a storage medium of the memory. That being one of the multiple options presented in [0147], like a hard disk), and the computation device controls the operation unit to send the second image to the data access unit and store the second image in the storage medium ([0064] lines 9-15 discloses that the memory stores images generated as a result of the mobile phone 200. Further [0062], discloses the performance of the super resolution processing by the image processing module, and as a result, by the processor. Subsequently, this generates the second image (correspondent to the claims). Thus, the second image, being an image generated as a result of the mobile phone 200 (being the computation circuit that comprised the operation circuit, each correspondent to the claims, respectively), it is understood that the second image is able to be stored int eh storage medium (each correspondent to the claims, respectively). Further, this transfer of information is from the processor to the memory, and as such utilizes the connection between the two, shown in Fig. 2. Disclosing the use of the data access circuit to provide the data to the storage medium (each correspondent to the claims respectively)).
In re to claim 8 [dependent on claim 1], Chaung teaches wherein: controlling, by the computation device which comprises the operation unit, (Fig. 2 shows that all systems used are comprised within the computation device, correspondent to the claims. See also Fig. 2 (220), which shows inclusion of the processor. Thus, it controls all actions performed within it and is comprised of the operation circuit (correspondent to the claims))
Chaung does not explicitly teach wherein: the performing of the subsequent operations on the intermediate result includes: controlling, by the operation unit, the primary operation module to add bias data to the intermediate result, and then performing an activation operation.
However, in a related field of endeavor, Navarrete Michelini teaches wherein: the performing of the subsequent operations on the intermediate result includes: controlling, by the operation unit, the primary operation module to add bias data to the intermediate result, and then performing an activation operation (Fig. 4 shows the addition of bias (see also [0040] which corroborates this). Additionally, this subsequently is included into the intermediate result due to the forward propagation of this bias by the nature of a convolutional neural network. Further, following the addition of bias, an output is produced. Thus, an activation operation is performed, which is understood to be the generation of the output).
The reasons for combination are the same as provided above.
In re to claim 9 [dependent on claim 8], Chaung teaches wherein: controlling, by the computation device which comprises the operation unit (Fig. 2 shows that all systems used are comprised with the computation circuit, correspondent to the claims. See also Fig. 2 (220), which shows inclusion of the processor. Thus, it controls all actions performed within it and is comprised of the operation circuit (correspondent to the claims))
Chuang does not explicitly teach wherein: the primary operation module includes a first operation unit, wherein the first operation unit includes a vector addition unit and an activation unit, controlling, by the computation device, the primary operation module to add bias data to the intermediate result, and then performing an activation operation include: controlling, by the operation unit, the vector addition unit to implement a bias addition operation of a convolutional neural network operation and perform elementwise addition on bias data and the intermediate result to obtain a bias result, and controlling, by the operation unit, an activation unit to perform an activation function operation on the bias result.
However, in a related field of endeavor, Navarrete Michelini teaches wherein: the primary operation module includes a first operation unit, wherein the first operation unit includes a vector addition unit and an activation unit (Fig. 4 and [0040] disclose the primary operation module performs the addition of bias onto intermediate vectors (correspondent to the claims). Further, as this is an action of addition by adding bias to said vectors, it is understood to disclose a vector addition unit. Additionally, as is shown in Fig. 4, there is an output from the bias adding portion of the convolution layer. This is understood to indicate the teaching of an activation unit, being the portion of the primary operation circuit that generates the output. With the addition and activation units (in combination) being understood (correspondent to the claims, each respectively) to be the first operation circuit) , controlling, by the computation device, the primary operation module to add bias data to the intermediate result, and then performing an activation operation include: controlling, by the operation unit, the vector addition unit to implement a bias addition (Fig. 4 and [0040] disclose the primary operation module performs the addition of bias onto intermediate vectors (correspondent to the claims). Further, as this is an action of addition by adding bias to said vectors is performed by the vector addition circuit (correspondent to the claims). Additionally, per Fig. 4, the activation operation is performed as well to generate an output. This is understood to be done by the primary operation module, as it comprises the first operation circuit (each correspondent to the claims respectively)) operation of a convolutional neural network operation and perform elementwise addition on bias data and the intermediate result to obtain a bias result (Fig. 4, shows that the neutral network is a convolutional neural network. Further, the addition of bias is to that of filtered elements from that of a feature image, as suggested by [0040] and Fig. 4. Thus, it is understood that element-wise addition is performed, as it occurs to filtered elements, being the Wijk block outputs. Subsequently, the bias addition generates a result, being the elements with added bias), and controlling, by the operation unit, an activation unit to perform an activation function operation on the bias result(Fig. 4 and [0040], disclose the output of the activation function operation (being the application of the function that is performed by the activation circuit, correspondent to the claims to produce an output of the bias adding portion of the convolutional layer). Thus, disclosing that the activation function operation is on that of the bias result (correspondent to the claims), as it is what is used to generate the output image (given that the output image is not simply the bias added elements)).
Navarrete Michelini, like Chuang, teaches the alteration of an input image’s resolution to that of a higher resolution. This being a preset design element of their respective systems.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective
filing date of the claimed invention to modify Chuang, to implement a convolutional neural network that includes the various elements required to perform convolutional operations, as taught by Navarrete Michelini, to arrive at the claimed invention discussed above. The motivation for the proposed modification would allow for the system to integrate the adaptability of a convolutional neural network to better address varied images of different image data. Allowing for the upscaling of images to better fit an individual image, as is done in Navarrete Michelini Fig. 2E’s upscaling apparatus system.
In re to claim 11 [dependent on claim 1], Chaung teaches wherein: controlling, by the computation device which comprises the operation unit, (Fig. 2, shows that all systems used are comprise the computation circuit, correspondent to the claims. See also Fig. 2 (220), which shows inclusion of the processor. Thus, it controls all actions performed within it and is comprised of the operation circuit (correspondent to the claims))
Chuang does not explicitly teach wherein: each of the plurality of secondary operation modules includes a second operation unit, further wherein the second operation unit includes a vector multiplication unit and an accumulation unit, controlling, by the computation device, the plurality of secondary operation modules to perform a convolution operation of input data and a convolution kernel in a convolutional neural network algorithm includes: controlling, by the operation unit, the vector multiplication unit to perform a vector multiplication operation of the convolution operation, and controlling, by the operation unit, the accumulation unit to perform an accumulation operation of the convolution operation.
However, in a related field of endeavor, Navarrete Michelini teaches wherein: each of the plurality of secondary operation modules includes a second operation unit, further wherein the second operation unit includes a vector multiplication unit and an accumulation unit (Fig. 4, depicts a convolutional layer, and thus, convolution is performed, this convolution is understood to be performed by the second operation module (being one of the plurality of secondary operation modules, correspondent to the claims). Further, as this is an action of convolution, multiplication and subsequent accumulation of the results (as a result of the multiplication that occurs) is disclosed. Thus, showing that there is a multiplication unit and an accumulation unit), controlling, by the operation unit, the plurality of secondary operation modules to perform a convolution operation of input data and a convolution kernel in a convolutional neural network algorithm includes (Fig. 4 shows that the convolutional neutral network is a convolutional neural network. As such, it includes a convolution kernel to perform the convolution of the convolutional neural network algorithm (correspondent to the claims)): controlling, by the operation unit, the vector multiplication unit to perform a vector multiplication operation of the convolution operation, and controlling, by the operation unit, the accumulation unit to perform an accumulation operation of the convolution operation (Fig. 4 depicts a convolutional layer, and thus, convolution is performed. This convolution is understood to be performed by the second operation module (correspondent to the claims). Further, as this is an action of convolution by a convolution network, multiplication and subsequent accumulation of the results occurs (as a result of the multiplication that occurs). Thus, showing that there is a multiplication circuit and an accumulation circuit).
Navarrete Michelini, like Chuang, teaches the alteration of an input image’s resolution to that of a higher resolution. This being a preset design element of their respective systems.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective
filing date of the claimed invention to modify Chuang, to implement a convolutional neural network that includes the various elements required to perform convolutional operations, as taught by Navarrete Michelini, to arrive at the claimed invention discussed above. The motivation for the proposed modification would allow for the system to integrate the adaptability of a convolutional neural network to better address varied images of different image data. Allowing for the upscaling of images to better fit an individual image, as is done in Navarrete Michelini Fig. 2E’s upscaling apparatus system.
As to claim 15, it is the system that performs the method of claim 1 and 8. As such it recites similar limitations to method claim 1 and 8, and is rejected for the same reasons as provided.
As to claim 16, it is the system that performs the method of claim 2. As such it recites similar limitations to method claim 2, and is rejected for the same reasons as provided.
As to claim 17, it is the system that performs the method of claim 3, in combination with claim 4. As such it recites similar limitations to method claims 3 and 4, and is rejected for the same reasons as provided.
As to claim 18, it is the system that performs the method of claim 5. As such it recites similar limitations to method claim 5, and is rejected for the same reasons as provided.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN M COOMBER whose telephone number is (571)270-0950. The examiner can normally be reached Monday - Friday 8:00am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Gregory Morse can be reached at (571) 272-3838. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN M COOMBER/Examiner, Art Unit 2663
/GREGORY A MORSE/Supervisory Patent Examiner, Art Unit 2698
Prosecution Insights