Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 08/13/2026 have been fully considered but they are not persuasive. On page 11, applicant argues that Kang does not disclose training apparatus or a training method for a differentiable model of an image signal processor, the model having differentiable modules configured
to perform a single image signal processing function, as required by claim 1. The Examiner respectfully disagrees. Every single image analysis has “a single image signal processing function” by default of analyzing any images. Further, there are differentiable modules as seen in the figure they are differentiable, since there are no further details in the claim language. The additional argument that claim 1 requires training of the first differentiable module comprises inputting, processing, calculating and updating features and not taught by Kang. The Examiner respectfully disagrees, there steps of “inputting”, “processing” and “calculating and updating” without any actual details of how, and what are being updated, then any module at any point can take an input, process, do some sort of calculating, and by running the system again updating. In other words the claims have general steps of training and inputting and outputting and therefore the same module can be pointed out as doing the same tasks. The order of these steps can be important with more detail on what is being updated, inputting calculated and processed as disclosed in claims 4 and 7 in combination for example for “using a numerical optimiser based on gradient descent by back-propagation of error.” Along with the limitations of claim 4. Otherwise the way the claims are currently reading, the claims are far too general to be allowable or to train a differentiable model of image better. For these reasons the same rejection applies.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 8-16, and 18-20 are rejected under 35 U.S.C. 103 as being unpatentable over Kang (US 2023/0071693) in view of Sanchit (US 2024/0311975).
As per claims 1, 11 and 20, Kang teaches, a training apparatus and method and a non-transitory computer readable for training a differentiable model of an image signal processor (Kang, ¶[0014] “obtaining a generative network comprising a differentiable activation layer” This represents a differentiable model), the image signal processor having a pipeline of separate image signal processing functions, wherein the differentiable model of the image signal processor comprises at least two differentiable modules (Kang, fig.4 represents pipeline process one after another of processing functions, and seen fig.6 at least two different functions ), each of the differentiable modules of the differentiable model of the image signal processor being configured to perform a respective single image signal processing function of the pipeline (Kang, fig.4 every step has a different function), the training apparatus comprising one or more processors configured to:
a degraded image (Kang, fig.2B 201 represents degraded image X) and train a first differentiable module of the differentiable model of the image signal processor to perform a first image signal processing function (Kang, fig.4, 410 then 420 “train first teacher network…” represents train a first differentiable module), whilst not training other differentiable modules of the differentiable model of the image signal processor (Kang, fig.4 training takes place at different times in the pipeline therefore whilst not training other differentiable modules such as 440 ), by iteratively: inputting, to the differentiable model of the image signal processor, a degraded image signal that represents a known degradation of the reference image (Kang, fig.4 410 receive degraded training image, which would be that degraded reference image, just Kang fails to mention and another reference below will address this ), the degradation being related to the first image signal processing function, processing the degraded image signal using the differentiable model of the image signal processor to produce a first processed image, said processing including using the first differentiable module to perform the first image signal processing function (Kang, fig.4, 420 is the first differentiable module to perform the first image signal processing function), calculating an error between the first processed image and the reference image by comparing the first processed image to the reference image (Kang, fig.1 130 loop represents an error between the first processed image and the reference image), and updating the first image processing function performed by the first differentiable module based on the calculated error without updating the image processing functions performed by other differentiable modules of the differentiable model of the image signal processor (Kang, fig.1 Restored image 120 represents updating the first image processing function performed by the first differentiable module based on the calculated error by going by 130 restoration network, and then 100 represents with without updating the image processing functions performed by other differentiable modules of the differentiable model of the image signal processor two different paths ).
Kang doesn’t clearly teach, receive a reference image.
However, Sanchit teaches, receive a reference image (Sanchit, fig.3, 301 Receive training images representing target, represents reference image, and in the same area see Abstract “Methods of training a machine learning model for image processing are described. A method of training includes utilizing as a learning objective a reduction or minimization of a combination of both an image loss and a classification loss.” ).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teachings of Kang with those of Sanchit to have the reference image ahead of a degraded image which would be obvious to have a reference image then make a degraded image from that, however Kang is listen about this.
The motivation would have been for the enhancement and make the image clearer or enable information from the image to the more readily discerned as taught by Sanchit in ¶[005].
As per claims 2 and 12, Kang in view of Sanchit teaches, the training apparatus of claim 1, wherein the first differentiable module comprises logic configured to perform a base image processing function, and/or a refinement function, and wherein the one or more processors are further configured to: update, as part of updating the first image processing function, the parameters of the refinement function and/or base image processing function based on the calculated error (Kang, fig.1 130 is a base image processing function there are no other details in the claim language, and this is in the calculated error in 130 that gets calculated).
As per claims 3 and 13, Kang in view of Sanchit teaches, the training apparatus of claim 1, wherein the one or more processors are further configured to update one or more parameters of the first image processing function by a first amount per iteration based on at least one first pre-set learning rate (Kang, ¶[0027] “a first output of a first teacher network and increases a second difference between the third output and a second output of the second teacher network” this represents the first image processing function by a first amount per iteration based on at least one first pre-set learning rate, since it is the first teacher network being developed, and the preset learning rate would then be set by this first teacher network).
As per claims 4 and 14, Kang in view of Sanchit teaches, the training apparatus of claim 1, wherein the one or more processors are further configured to train a second differentiable module of the differentiable model of the image signal processor to perform a second image signal processing function different from the first image signal processing function, whilst not training other differentiable modules of the differentiable model of the image signal processor (Sanchit, fig.3 305 would be the second while not training the others), the one or more processors configured to train the second differentiable module by iteratively: inputting, to the differentiable model of the image signal processor, a second degraded image signal that represents a second known degradation of the reference image (Sanchit, fig.3 301 is the reference image and training in 305 second image processing function); processing the second degraded image signal using the differentiable model of the image signal processor to produce a second processed image, said processing including using the second differentiable module to perform the second image signal processing function; calculating an error between the second processed image and the reference image by comparing the second processed image to the reference image (Sanchit, fig.4 403, determine image loss target represents the error ); and updating the second image processing function performed by the second differentiable module based on the calculated error without updating the image processing functions performed by other differentiable modules of the differentiable model of the image signal processor (Sanchit, fig.3-4, the process is the same exact thing and as can be seen the branches going 304-306.. multiple branches and different calculations represent the second branch being trained individually. And each function as can be seen is different. And looping around from 405 back to 402 in fig.4 represents the updating).
As per claims 5 and 15, Kang in view of Sanchit teaches, the training apparatus of claim 4, wherein after each of the at least two differentiable modules have been independently trained, the one or more processors are configured to train both differentiable modules simultaneously by: receiving a reference image, and iteratively: inputting, to the first differentiable module, a third degraded image signal that represents a third known degradation of the reference image (Sanchit, fig.3 306 would represent the third module, and depending on interpretation they are trained at different times and at the same time ); processing, using the first differentiable module, the third degraded image signal by performing the first image signal processing function to produce a partially processed image signal; inputting, to the second differentiable module, the partially processed image signal; processing, using the second differentiable module, the partially processed image signal by performing the second image signal processing function to produce a third processed image; calculating an error between the third processed image and the reference image by comparing the third processed image to the reference image (Sanchit, fig.4, 403, image loss would be the error); and updating the first and/or second image processing functions performed by the respective first and/or second differentiable modules based on the calculated error (Sanchit, fig.3-4, the process is the same exact thing and as can be seen the branches going 304-306.. multiple branches and different calculations represent the third branch being trained individually. And each function as can be seen is different. And looping around from 405 back to 402 in fig.4 represents the updating).
As per claims 6 and 16, Kang in view of Sanchit teaches, the training apparatus of claim 4, wherein the one or more processors are further configured to fix the parameters of the first differentiable module or the second differentiable module that is not being trained while the other of the first or a further differentiable module is trained (Kang, ¶[0064] “FIGS. 2A and 2B illustrate an example of an architecture and a training process of a generative network. Referring to FIG. 2A, illustrated is an architecture of a generative network G 210.” One is being trained, and then the others the restored image 203 is being fixed ).
As per claims 8 and 18, Kang in view of Sanchit teaches, the training apparatus of claim 1, wherein the at least two differentiable modules are any two of a demosaicing module, a sharpener module, a black-level subtraction module, a spatial denoiser module, a global tone mapping module, a channel gain module, an automatic white balance, or a colour correction module (Kang, ¶[0057] “For example, for a scene class of people (when the photo is of a person or group of people) any adjustment of the saturation (color) may add less color to achieve an optimum in comparison to an amount of color added for a scene class of food (when the photo is of food)” this represents colour correction module).
As per claims 9 and 19, Kang in view of Sanchit teaches, the training apparatus of claim 1, wherein the one or more processors are further configured to stop the iterative process when the calculated error is less than a threshold (Sanchit, ¶[0066] “For example a minimum threshold may be when there is a visually discernible degradation of the image and a maximum threshold may be when the image starts to become unrecognizable or when a significant amount of detail starts to become lost.” This represents stopping after the error of too much degradation is above a threshold ).
As per claim 10, Kang in view of Sanchit teaches, the training apparatus of claim 1, wherein the degraded image signal that represents a known degradation of the reference image, represents a degradation produced by a modular capture model that is based on the characteristics of a physical image capture apparatus (Sanchit, ¶[0044] “By way of example, where the computer processing system 200 is the client system 130 it may include a display 218 (which may be a touch screen display), a camera device 220, a microphone device 222 (which may be integrated with the camera device), a pointing device 224 (e.g. a mouse, trackpad, or other pointing device), a keyboard 226, and a speaker device 228.” This would represent a modular capture model that is based on the characteristics of a physical image capture apparatus which is being represented by camera device 220, and becomes the reference image).
Allowable Subject Matter
Claims 7 and 17 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The subject matter applies backpropagation to arbitrary computational graphs and converting formerly un-trainable steps into trainable differentiable modules. This step allows combining neural network flexibility with the rigor of traditional algorithmic solvers. This further solves the ability to train differentiable modules using gradient descent by back-propagation of error within a unified processor framework became notable not because of the underlying algorithm, but because it enables end-to-end, modular, and domain-aware learning, specifically solving the problem of incorporating "non-differentiable" operations into a differentiable system. There was not a close prior art with this subject matter prior to the effective filing date of the current application.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SANTIAGO GARCIA whose telephone number is (571)270-5182. The examiner can normally be reached Monday-Friday 9:30am-5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SANTIAGO GARCIA/Primary Examiner, Art Unit 2673
/SG/
Prosecution Insights