DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This is in response to applicant's amendment/response filed on 06/23/2026, which has been entered and made of record.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Naccari et al (US 20230078190 A1).
Regarding claim 1, Naccari discloses a method (Naccari [0005], “method”), comprising:
storing weight data configured to weigh on image data (Naccari [0052], “Encoding is carried out using the weights stored in the model”; [0053], “A normalised weight W.sub.N for each image area is determined”);
receiving first data representative of a first portion of an image (Naccari [0053], “for each image area (a first data representing a first received area/portion of an image)”);
determining, based on a location of the first portion within the image, a first quantization level (Naccari [0056], “determining the weight W.sub.current for each area (area is interpreted as comprising a portion at a given location in an image) of a current image”; [0070], “a discrete quantisation level Δ.sub.area used for compression (comprising a first quantization level)”); and
quantizing the first data according to the first quantization level (Naccari [0070], “a discrete quantisation level Δ.sub.area used for compression (compression/quantizing)”);
Naccari does not explicitly disclose a location of the first portion within the image.
However, Naccari suggests a location of the first portion within the image (Naccari [0056], “determining the weight W.sub.current for each area (area is interpreted as comprising a portion at a given location in an image) of a current image”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to utilize area information as location data for portions of an image. This would have been done to generate a high quality compressed image. See, for example, Naccari [0025], “a photograph or a video, different areas of the image may be compressed at different levels. For example, a larger area of similar, or identical color, such as sky, may be compressed at a higher level, without losing quality perceived by a viewer of the display device 20, than an area with sharp boundaries between different colors providing high detail.”).
Regarding claim 15, Naccari discloses an apparatus (Naccari fig. 2), comprising:
a pair of augmented reality glasses (Naccari [0020], “enhanced reality wearable glasses or other head-mounted display system”), having:
a digital camera configured to capture an image of a field of view (Naccari [0020], “video cameras”); and
a processing device configured to perform an analysis of the image using an artificial neural network having weight data (Naccari [0052], “Encoding is carried out using the weights stored in the model”; [0053], “A normalised weight W.sub.N for each image area is determined”);
wherein the processing device is further configured to apply different quantization levels to data from different regions of the image (Naccari [0056], “determining the weight W.sub.current for each area (areas interpreted as comprising different regions in an image) of a current image”; [0070], “a discrete quantisation level Δ.sub.area used for compression (comprising a first quantization level)”), and
apply the different quantization levels to the weight data in weighing on the data from the different regions respectively (Naccari [0070], “a discrete quantisation level Δ.sub.area used for compression (compression/quantizing levels for each area respectively)”).
Naccari does not explicitly disclose a digital camera.
However, Naccari suggests a digital camera (Naccari [0020], “video cameras”; [0021], “the sources may be … video streams (interpreted as reading on digital image data)”).
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to utilize a digital camera, as suggested by Naccari, to capture image data for processing. This would have been done to optimize processing by eliminating the need for processing by eliminating analog to digital processing from a non-digital camera.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Naccari in view of Kim et al (US 20240104912 A1).
Regarding claim 8, Naccari discloses a device (Naccari fig. 1), comprising:
receive first data representative of a first portion of the image (Naccari [0053], “for each image area (a first data representing a first received area/portion of an image)”);; and
determine, based on a location of the first portion within the image, a first quantization level (Naccari [0056], “determining the weight W.sub.current for each area (area is interpreted as comprising a portion at a given location in an image) of a current image”; [0070], “a discrete quantisation level Δ.sub.area used for compression (comprising a first quantization level)”);
voltage drivers (Naccari [0019], “circuits” (comprising voltage drivers to drive the circuits)); and
a logic circuit (Naccari [0019], “circuits, and processes”) configured to:
Naccari does not disclose
an array of memory cells programmable in a first mode to support multiplication and accumulation
program, using the voltage drivers and in the first mode, first memory cells in the array to store weight data of an artificial neural network trained to analyze an image
However, Kim discloses
an array of memory cells programmable in a first mode to support multiplication and accumulation (Kim [0148], “the processing element array 100 is configured to include a plurality of processing elements (PE1 . . . ) 110 configured to calculate node data of an artificial neural network and weight data of a connection network. Each processing element may include a multiply and accumulate (MAC) operator”);
program, using the voltage drivers and in the first mode, first memory cells in the array to store weight data of an artificial neural network trained to analyze an image (Kim [0143], “the algorithms applied to the first and second models based on the artificial neural network, and the operating characteristics of the NPU 1000, … the weight values loaded into the internal memory (store weight data of an artificial neural network trained to analyze an image)”; [0168], “The NPU controller 300 may control (program) to induce at least one processing element to classify the object in the image (analyzing an image) using the first model”);
It would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to modify Naccari with Kim to utilize an NPU to perform MAC operations to analyze image objects. This would have been done to improve image quality. See, for example, Kim [0221], “provide an image having an optimal quality according to characteristics of an object by using a plurality of independent neural network-based models”
Allowable Subject Matter
Claims 2-7, 9-14 and 16-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Regarding claim 2, none of the prior art of record, alone or in combination, disclose the claim as recited as a whole.
Claims 3-7 are allowed for depending from claim 2.
Regarding claim 9, none of the prior art of record, alone or in combination, disclose the claim as recited as a whole.
Claims 10-14 are allowed for depending from claim 9.
None of the prior art of record, alone or in combination, disclose the claim as recited as in Claims 16-20.
Response to Arguments
Applicant's arguments filed on 06/23/2026 have been fully considered but they are not persuasive.
On pgs. 1-3 applicant argues:
Claims 1 and 15 were rejected over Naccari (20230078190). The rejection is respectfully traversed.
Claim 8 was rejected over Naccari and Kim (20240104912). The rejection is respectfully traversed.
Applicant’s invention as claimed is directed to techniques of image processing at different quantization levels adapted according to a perception or vision characteristics of human ocular focus, where what is in the center of a field of vision is seen clearer than what is on the periphery of the field of vision.
As claimed by Applicant, quantization of both image data and weight data to be applied to weigh the image data is configured at multiple levels to emulate the perception or vision characteristics of human ocular focus.
For example, image data and corresponding weight data for an image region of more interest (e.g., the center region of an image to be analyzed by an artificial neural network to recognize, extract, classify, or identify objections) is applied quantization simultaneously at a level that is more accurate than the quantization level applied to an image region of less interest (e.g., a peripheral region of the image).
For example, a same weight matrix can be configured to be applied to weigh a unit of image data to generate weighted and summed inputs to a set of artificial neurons. Such a unit of image data can be for a block of pixels of a predetermined number of rows and a predetermined number of columns, where the blocks of pixel can be in any of the different regions (e.g., center region, intermediate region, transition region, peripheral region). The same weight matrix of high accuracy can be applied to different units of image data from the different regions of an image to perform the computation of the artificial neural network at the same accuracy level.
To emulate the perception or vision characteristics of human ocular focus, the weight matrix can be quantized to generate a plurality of quantized weight matrices at different levels of accuracy. For example, data elements in a quantized weight matrix at a high level of accuracy can be each represented by integer numbers of a fixed width of a high number of bits; and data elements in a quantized weight matrix at a low level of accuracy can be each represented by integer numbers of a fixed width of a low number of bits. Thus, a same weight can be represented by different integer numbers of different bit widths configured for different quantization levels respectively, although the ratio between an integer number representative of a quantized weight at a given level of accuracy and the range of possible integer numbers representative of different quantized weights at the same level of accuracy can be the same across the quantization levels.
For example, quantization of a number for a level of accuracy can be performed efficiently via bitwise shifting to remove less significant bits and retain a predetermined number of most significant bits; and quantization configured for different levels of accuracy can be configured to retain different numbers of most significant bits and thus different bit widths.
When a unit of image data is from an image region that is of high interest (e.g., center region), the unit of image data can be quantized at a high level of accuracy to generate a quantized unit of image data, where each integer number has a high bit width. A quantized weight matrix at the same high level of accuracy can be selected and used to weigh the quantized unit of image data. Multiplication and accumulation can be applied to the quantized unit of image data and the quantized weight matrix, having matching high accuracy levels, in generating weighted sum of inputs. The result of the multiplication and accumulation (e.g., as inputs to a set of artificial neurons) has a high level of accuracy, which corresponds to the high quantization level applied to both the weight matrix and the unit of image data.
In contrast, when a unit of image data is from an image region that is of low interest (e.g., peripheral region), the unit of image data can be quantized at a low level of accuracy to generate a quantized unit of image data, where each integer number has a low bit width. A quantized weight matrix at the same matching level of accuracy can be selected and used to weigh the quantized unit of image data. Multiplication and accumulation can be applied to the quantized unit of image data and the quantized weight matrix, having matching low accuracy levels, in generating weighted sum of inputs. The result of the multiplication and accumulation has a low level of accuracy corresponding to the low quantization level applied to both the weight matrix and the unit of image data.
Thus, the computation results have an accuracy characteristics emulating the perception or vision characteristics of human ocular focus: the results for the image data computed for the region of interest (e.g., center region) are more accurate (e.g., corresponding to clearer vision) than the results for the image data computed for region of less interest (e.g., peripheral region).
Since the computations for image data in regions of less interest are configured to use a smaller number of bits, the performance of the computations consumes less energy, which leads to savings in overall energy consumption.
Similar to perception through human ocular focus, the quality of the analysis of the artificial neural network (e.g., in object detection, extraction, identification, classification) can be degraded in the regions of less interest (e.g., peripheral region of the image representative of what is in the field of vision of an eye of a user).
The techniques to reduce energy consumption by reducing the computation accuracy through quantization in image regions of less interest can be applied in context-aware applications, such as augmented reality (AR) presented via smart glasses.
Augmented reality (AR) glasses can be configured to capture and analyze an image of the field of view in front of a user. Objects in the image can be analyzed to recognize objects; and information about or related to the recognized objects can be presented to the user using the glasses to augment the reality seen through the glasses.
Since a typical user is less concerned about the objects in their peripheral vision, degrading the accuracy in recognizing the objects appearing in the peripheral vision in exchange for reduced energy consumption can be beneficial and desirable.
Applicant’s invention as directed to techniques of image processing at different quantization levels adapted according to a perception or vision characteristics of human ocular focus, where what is in the center of a field of vision is seen clearer than what is on the periphery of the field of vision, is captured via the claimed elements of the independent claims, such as claim 1 set forth below.
Examiner respectfully disagrees,
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., for example, “unit of image data can be for a block of pixels of a predetermined number of rows and a predetermined number of columns, where the blocks of pixel can be in any of the different regions (e.g., center region, intermediate region, transition region, peripheral region). The same weight matrix of high accuracy can be applied to different units of image data from the different regions of an image to perform the computation of the artificial neural network at the same accuracy level.”) are not recited in rejected claims 1, 8 or 15. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993).
On pg. 4, applicant argues:
Applicant’s invention as directed to techniques of image processing at different quantization levels adapted according to a perception or vision characteristics of human ocular focus, where what is in the center of a field of vision is seen clearer than what is on the periphery of the field of vision, is captured via the claimed elements of the independent claims, such as claim 1 set forth below.
1. A method, comprising:
storing weight data configured to weigh on image data;
receiving first data representative of a first portion of an image;
determining, based on a location of the first portion within the image, a first quantization level; and
quantizing the first data according to the first quantization level.
The prior art reference Naccari (20230078190) fails to teach or disclose techniques of image processing at different quantization levels adapted according to a perception or vision characteristics of human ocular focus, where what is in the center of a field of vision is seen clearer than what is on the periphery of the field of vision, as claimed by Applicant.
Rather, Naccari is limited to disclosing controlling compression of image data by a quality controller include obtaining a desired target number of bits to be generated from compression of a current image area using a predetermined compression protocol.
Determining a calculated quantisation level based on the desired number of bits using a predetermined relationship between the number of bits and quantisation level. Selecting a discrete quantisation level from a plurality of predetermined discrete quantisation levels based on the calculated quantisation level. Determining a predicted number of bits that would result from compression of the current image area at the selected discrete quantisation level using the predetermined relationship. Determining whether the predicted number of bits exceeds the desired number of bits and, if not, providing to an encoder information to enable the encoder to determine a set of compression parameters associated with the selected discrete quantisation level.
The prior art reference, Kim (20240104912), fails to rectify the shortcomings of Naccari. As Kim is limited to disclosing receiving an image including an object. Classifying at least one object in the image using a first model on the basis of an artificial neural network configured to classify the at least one object by inputting the image. Obtaining an image having improved quality according to the at least one object by inputting the image in which the at least one object is classified by using at least one model among a plurality of second models on the basis of an artificial neural network configured to output a specialized processing applied image according to a particular object by inputting the received image.
Accordingly, claims 1 and 15 are patentable over Naccari (20230078190). And, claim 8 is patentable over Naccari and Kim (20240104912).
Examiner respectfully disagrees,
Naccari discloses
image processing at different quantization levels (Naccari [0056], “determining the weight W.sub.current for each area (areas interpreted as comprising different regions in an image) of a current image”; [0070], “a discrete quantisation level Δ.sub.area used for compression (comprising a first quantization level)”).
The invention as recited in claims 1, 8 and 15 does not recite, “different quantization levels adapted according to a perception or vision characteristics of human ocular focus, where what is in the center of a field of vision is seen clearer than what is on the periphery of the field of vision”
Applicant’s arguments with respect to Kim are also not persuasive because applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references.
Kim is relied upon to disclose the following limitations. However, the applicant’s arguments do not pertain to these limitations.
an array of memory cells programmable in a first mode to support multiplication and accumulation (Kim [0148], “the processing element array 100 is configured to include a plurality of processing elements (PE1 . . . ) 110 configured to calculate node data of an artificial neural network and weight data of a connection network. Each processing element may include a multiply and accumulate (MAC) operator”);
program, using the voltage drivers and in the first mode, first memory cells in the array to store weight data of an artificial neural network trained to analyze an image (Kim [0143], “the algorithms applied to the first and second models based on the artificial neural network, and the operating characteristics of the NPU 1000, … the weight values loaded into the internal memory (store weight data of an artificial neural network trained to analyze an image)”; [0168], “The NPU controller 300 may control (program) to induce at least one processing element to classify the object in the image (analyzing an image) using the first model”).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JITESH PATEL whose telephone number is (571)270-3313. The examiner can normally be reached 8am - 5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said A. Broome can be reached at (571) 272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JITESH PATEL/Primary Examiner, Art Unit 2612
Prosecution Insights