DETAILED ACTION
Response to Amendment
Claims 1-22 were previously pending. Applicant’s amendment filed August 12, 2026, has been entered in full. Claims 1-4, 6-15, 17, and 19-22 are amended. No claims are added or cancelled. Accordingly, claims 1-22 are now pending.
Response to Arguments
Applicant notes a previous indication of allowable subject matter (Remarks filed August 12, 2026, hereinafter Remarks: Page 9). Examiner notes that Applicant has made intervening amendments that raise new issues (see rejections below) so the previously-indicated claims are no longer allowable.
Applicant “traverses the Examiner’s characterization of the plain meaning of the term ‘raw imagery’” and states that the term should be interpreted under BRI (Remarks: Page 9). Applicant’s position is respectfully unclear. Interpretation under BRI requires determining the plain meaning of a term, which Examiner’s previous action explained. If Applicant disagrees with this plain meaning, then they are invited to explain why. For example, by proposing an alternative plain meaning. Applicant’s traversal without explanation is respectfully non-persuasive and the previous interpretation of “raw imagery” under BRI is maintained.
Applicant argues that amendments to the claims have overcome the previous objections to claims 9 and 19 (Remarks: Page 9). Examiner agrees. The previous objections to these claims are withdrawn.
Applicant argues that the amendments to the claims have overcome the previous rejection of claims 6 and 7 under 35 U.S.C. 112(b) (Remarks: Page 9). Examiner agrees. The previous rejection under 35 U.S.C. 112(b) is withdrawn.
Applicant traverses the previous rejections under 35 U.S.C. 103, arguing that the previously-cited Zhang and other prior art references do not teach all elements of the amended claims (Remarks: Pages 9-12). In particular, Applicant argues that Zhang lacks what is termed (a) a simultaneous split architecture and (b) a system response generation that jointly depends on generated higher-quality imagery and tags derived from lower quality imagery (Remarks: Page 12). Examiner respectfully disagrees.
Regarding feature (a), Applicant argues that the specification describes controller and PLD processing “substantially simultaneously” at par. [0065] (as filed) and asserts that “Zhang is silent regarding the timing of the processing paths” (Remarks: Page 12). Examiner respectfully disagrees. The term “substantially” “is a broad term.” MPEP 2173.05(b), Subsection III.D. The specification does not expressly define or limit what specifically is meant by “substantially,” but the term is used at par. [0065] to describe the degree to which the parallel processing of raw image data 511 by both controller 432 and PLD 400 illustrated in Fig. 5 is simultaneous. This suggests that the broadest reasonable interpretation (BRI) of “substantially simultaneously” includes parallel processing configurations where the same data is processed in parallel. I.e., by inputting the same raw image data to multiple processors in parallel, the same data is being processed substantially simultaneously.
Zhang discloses the same type of parallel processing. For example, compare Fig. 2 in Zhang to Fig. 5 of the instant application (reproduced below).
PNG
media_image1.png
200
400
media_image1.png
Greyscale
PNG
media_image2.png
174
400
media_image2.png
Greyscale
Fig. 5 of instant application
(exemplary images omitted)
Fig. 2 of Zhang
Both Zhang and the processing described at par. [0065] take the same raw data and input it in parallel to conventional, human-quality image processing (controller 432 in Fig. 5 and ISP 230 in Fig. 2 of Zhang) and engine-quality image processing (PLD 400 in Fig. 5 and blocks 230 and 240 in Fig. 2 of Zhang) pipelines, perform processing in parallel, and then merge results (at OS 442 in Fig. 5 and at display 230 in Fig. 2 of Zhang). In view of at least this evidence, the parallel processing of Zhang falls within the scope of “substantially simultaneously” as interpreted under BRI.
Regarding feature (b), Applicant argues that Zhang does not teach generating a system response based on both higher-quality imagery and tags generated from lower-quality imagery (Remarks: Page 12). Examiner respectfully disagrees. As illustrated in Fig. 2, the display at step 230 takes input from both ISP 210 (which produces higher/human-quality imagery) and AI 240 (which produces tags). Furthermore, par. [0022] of Zhang states that higher/human-quality imagery from ISP 210 is displayed along with additional or enhanced information from AI 240. Note from par. [0021] that the AI functions to perform tagging functions such as object detection. For at least these reasons, Zhang describes generating a system response based on both the generated second/high-quality imagery and the tags.
Claim Interpretation
Claims are given their broadest reasonable interpretation (BRI) during examination. MPEP 2111. Under BRI, the words of a claim are given their plain meaning, unless such meaning is inconsistent with the specification. MPEP 2111.01, Subsection I. The plain meaning of a term is the ordinary and customary meaning given to the term by those of ordinary skill in the art at the relevant time. Id.
Claims 1 and 12, and their dependents, recite the term “raw imagery”.
Within the art of image analysis, the plain meaning of raw imagery (also sometimes written in uppercase letters as RAW imagery) is image data that is unprocessed and received directly from a digital camera. For example, direct sensor readings that have not been demosaiced, color-corrected, etc. Raw images cannot be viewed directly. Instead, they must be further processed to generate a viewable image. This interpretation is not inconsistent with the specification.
Claims 1 and 12, and their dependents recite the term “substantially simultaneously”. The word “substantially” “is a broad term.” MPEP 2173.05(b), Subsection III.D. The specification does not expressly define or limit what specifically is meant by “substantially simultaneously”, but the term is used at par. [0065] (as filed) to describe the degree to which the parallel processing of raw image data 511 by both controller 432 and PLD 400 illustrated in Fig. 5 is simultaneous. This suggests that the broadest reasonable interpretation (BRI) of “substantially simultaneously” includes parallel processing configurations where the same data is processed in parallel. I.e., by inputting the same raw image data to multiple processors in parallel, the same data is being processed substantially simultaneously.
Claim Objections
Claim(s) 14 is/are objected to because of the following informalities:
In claim 14, last line, “a second quality” should be “the second quality”
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 2-11 and 13-22 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Claim 2 recites that “the edge PLD is configured to: power, wake, depower, or sleep the controller … based, at least in part, on the one or more image tags and/or the generated first quality imagery.” This limitation is not adequately described in the original disclosure.
Claim 2 depends from claim 1, which has been amended to require that the PLD and the controller “are configured to process the raw imagery substantially simultaneously.” Such an embodiment is described at Fig. 5 and par. [0065] of the specification (as filed). Par. [0065] states that, in such embodiments, “controller 432 and/or system 430 may be powered and/or awake” and thus provide all processing paths shown in Fig. 5. Indeed, it would apparently be necessary for the controller to be awake and powered in order to perform processing substantially simultaneously with the PLD. However, this differs from what is claimed. Claim 2 recites powering or waking the controller based on tags and/or first quality imagery produced by the PLD. But how can the controller perform processing “substantially simultaneously” with the PLD if it is powered or woken based on outputs from the PLD’s processing? I.e., the specification does not (and logically cannot) describe a controller that both (a) processes raw imagery “substantially simultaneously” to a PLD and (b) is powered or woken in response to outputs from the PLD’s processing of that raw data. Claim 2 lacks adequate written description because it requires substantially simultaneous processing by the controller and PLD yet recites waking or powering the controller in response to outputs from the PLD, and the specification does not describe such an embodiment.
Furthermore, as discussed above, the claims appear to be directed to the embodiment described in par. [0065] of the specification (as filed) where PLD 400 and controller 432 process raw data substantially simultaneously. Par. [0065] mentions the controller being in an awake and powered state but does not describe any change in power state (i.e., powering, waking, depowering, or sleeping) of controller 432. Nor does par. [0065] describe the PLD 400 causing such a state change in the controller. Par. [0064] appears to contain the only mention of controller 432 being in a depowered or sleep state within the specification but, again, there is no mention of a state change caused by the PLD 400. Also, as explained above, the depowered or sleep state described at par. [0064] is incompatible with the description of substantially simultaneous processing described at par. [0065] and recited in the amended independent claims. The remainder of the specification mentions changing the power state (i.e., powering, waking, depowering, or sleeping) of electronic system 430. See, for example, par. [0063]. As illustrated in Fig. 4, PLD 400 and controller 432 are separate sub-components of electronic system 430. The description of powering, waking, depowering, or sleeping the overall electronic system 430 is more general than, and does not adequately describe, the specific powering, waking, depowering, or sleeping of the controller sub-component by the PLD sub-component as recited in currently presented claim 2.
Claim 13 recites similar limitations that also lack adequate written description for substantially the same reasons as claim 2. Claims 3-11 and 14-22 also lack adequate written description at least because they include the limitations of claim 2 or claim 13.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2-11 and 13-22 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 2 recites that “the edge PLD is configured to: power, wake, depower, or sleep the controller … based, at least in part, on the one or more image tags and/or the generated first quality imagery.” Claim 2 depends from claim 1, which states that the one or more image tags and the first quality imagery are generated by the PLD’s processing of raw image data and that the PLD and the controller process the raw image data substantially simultaneously.
The scope of claim 2 is unclear because it apparently recites contradictory and mutually exclusive requirements that cannot both be satisfied. For the controller to process the raw data “substantially simultaneously” with the PLD, the controller must be in a powered or awake state during processing of the raw data by the PLD. However, for the PLD to power or wake the controller based on the outputs of its processing, the controller must be in a depowered or asleep state during processing of the raw data by the PLD. Claim 2 is indefinite because it apparently requires the controller to be both powered/awake and depowered/asleep during the same processing by the PLD, yet these two states are mutually exclusive and cannot exist at the same time.
Claim 13 recites similar limitations and is also indefinite for substantially the same reasons as claim 2. Claims 3-11 and 14-22 are also indefinite at least because they include the limitations of claim 2 or claim 13.
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claim 19 is rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends.
Claim 12 requires “generating, via the controller, a response based, at least in part, on the generated second quality imagery and at least one of the one or more image tags generated using the first quality imagery by the edge PLD”
Claim 19, which depends upon claim 12, requires “generating by the controller a system response based, at least in part, on the generated second quality imagery and at least one of the one or more tags and/or the generated first quality imagery provided by the edge PLD.”
Claim 19 does not appear to provide any further limitations beyond what is already required by claim 12. Note that any response generated by the system of claim 12 is a “system response” and any response generated based on the tags is necessarily based on the generated first quality imagery because that is what is used to generate the tags. Claim 19 fails to comply with the requirements of 35 U.S.C. 112(d) because it does not further limit the subject matter of claim 12, upon which it depends.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1 and 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over ‘Zhang’ (CN 111355936 A; cited and copy provided in “parent” application 18/464,175).
Regarding claim 1, Zhang teaches an electronic system (e.g., Figure 2) comprising:
an edge programmable logic device (PLD) (e.g., [0012], an edge computing integrated circuit; see Note Regarding PLD below), wherein the edge PLD comprises a plurality of programmable logic blocks (PLBs) configured to implement an image engine preprocessor (Fig. 2, pre-processor 230) of the edge PLD and an image engine (Fig. 2, AI 240) of the edge PLD, wherein the edge PLD is configured to:
receive raw imagery Fig. 2, raw1) provided by an imaging module of the electronic system (Fig. 2, image sensor 200) via a raw image pathway of the electronic system (Fig. 2, arrow from 200 to 230),
generate, via the image engine preprocessor of the edge PLD (Fig. 2, pre-processor 230), first quality imagery by using the received raw imagery (Fig. 2, raw2; see Figs. 4 and 5 for details of the pre-processing that generates the raw2 first-quality imagery), and
generate, via the image engine of the edge PLD (Fig. 2, AI 240) using the first quality imagery (e.g., Fig. 2, first quality imagery raw2 is used by AI 240), one or more image tags (e.g., [0021], functions of AI engine may include object detection and recognition, both of which produce tags indicating an object type and/or location); and
a controller (Fig. 2, lower branch including ISP 210 and display module 220) configured to receive the raw imagery provided by the imaging module via the raw image pathway (e.g., [0022], Fig. 2, raw1 data passed along arrow from image sensor 200 to ISP 210), the controller being configured to:
generate second quality imagery by using the received raw imagery (e.g., [0022], Fig. 2, RGB imagery produced by ISP);
receive the one or more image tags from the edge PLD (e.g., Fig. 2, arrow from AI 240 to display module 220; e.g., [0022], results of AI processing – i.e., the one or more tags from the edge PLD – are output to display unit 220 as additional or enhanced information for enhanced display), and
generate a system response (e.g., [0022], the display provided by display unit 220) based, at least in part, on the generated second quality imagery (e.g., [0022], Fig. 2, raw imagery processed through ISP 210 – i.e., the second quality imagery – is displayed) and at least one of the one or more image tags generated using the first quality imagery by the edge PLD (e.g., [0022], Fig. 2, additional information from AI processing 240 – i.e., the tags generated using the first quality imagery by the edge PLD – are also output to the display unit for enhanced display),
wherein:
the first quality imagery comprises one or more of a lower resolution, a lower frame rate, a lower bit depth, a lower color fidelity, a narrower dynamic range, and/or a relatively lossy compressed state relative to the second quality imagery (e.g., [0028]-[0031], Fig. 5, channel selection combiner 450 uses only a subset of color information – i.e., only Green, Blue, Red, or Blue + Red – to generate the first/engine/raw2 quality imagery, which has lower color fidelity than the full RGB provided by the second quality imagery produced by the ISP; Note that an RGB image is illustrated at Fig. 3, element 310; Therefore, the first quality imagery has at least a lower color fidelity and/or a relatively lossy compressed state relative to the second quality imagery), and
the edge PLD and the controller are configured to process the raw imagery substantially simultaneously (Note the BRI provided in Claim Interpretation above), wherein the second quality imagery is generated by the controller while the image tags are generated by the edge PLD (e.g., Fig. 2, the processing of the raw data by the controller to generate second quality imagery at steps 210 and 230 takes place in parallel, when, and substantially simultaneously with the processing by the PLD to generate image tags at steps 230 and 240).
Note Regarding PLD. Zhang teaches that its system is preferably implemented as an integrated circuit chip for high-efficiency edge computing applications ([0012]). Zhang does not explicitly state that its system is implemented as a programmable logic device (PLD), such as an FPGA.
However, Zhang does that advantages of its approach include lower cost, lower power consumption, and higher speed (i.e., higher efficiency) via reduced signal bandwidth ([0037]). Zhang teaches that such reduced signal bandwidth is highly-required in hardware implementations such as FPGA silicon chips ([0037]).
Examiner notes that an FPGA is a specific type of PLD, which includes PLBs configured to implement processing functions. Therefore, a modification of Zhang to use an FPGA would fall within the scope of the claimed invention.
Given that Zhang suggests applying its techniques to FPGA silicon chips, and that such an FPGA silicon chip would qualify as the type of high-efficiency integrated circuit chip preferred by Zhang, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Zhang to implement its electronic system using an FPGA with the reasonable expectation that this would result in a system that provided the type of high-efficiency computing preferred by Zhang. This technique for improving the system of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Zhang.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang to obtain the invention as specified in claim 1.
Regarding claim 12, Examiner notes that the claim recites a method that is substantially the same as the method performed by the system of claim 1. Zhang teaches the system of claim 1 (see above). Accordingly, claim 12 is also rejected under 35 U.S.C. 103 as being unpatentable over Zhang for substantially the same reasons as claim 1.
Claim(s) 2-5, 8-10, 13-14, 17, and 19-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang as applied above, and further in view of ‘Srivastava’ (US 2019/0087690 A1).
Regarding claim 2, Zhang teaches the electronic system of claim 1.
Zhang further teaches that the controller is configured to generate human-quality imagery using the received raw imagery (e.g., [0022], Fig. 2, ISP 210 component of controller generates RGB imagery using received raw imagery; e.g., [0023], ISP’s algorithm is optimized for human vision).
Zhang does not explicitly teach the PLD being configured to power, wake, depower, or sleep the controller (Note the ‘112(b) rejection) and/or authenticate or deauthenticate a user access to the electronic system based, at least in part, on the one or more image tags and/or the generated first quality imagery.
However, Srivastava does teach that display devices may be made more power-efficient by only being powered on when a face is detected (e.g., [0001]). Specifically, Srivastava uses a low-power device to perform face detection (i.e., a type of object detection) and, upon detecting a face, powers-up a high-powered controller for tasks such as validation of a detected face or presentation of a login prompt (e.g., [0020], Figs. 1 and 3). The face detection can be seen as a “tag” on an input image and the powering of the controller and/or authentication of user access via a login prompt are based, at least in part, on the tag.
Zhang uses its image engine to perform object detection (e.g., [0021]) and seeks to achieve a reduction in power consumption (e.g., [0037]). Srivastava’s techniques provide a reduction in power consumption by selectively powering up a display, a face verification controller, and/or an authentication controller only when faces are detected.
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the system of Zhang with the face detection tag-based powering and/or authenticating of Srivastava in order to improve the system with the reasonable expectation that this would result in a system that achieved lower power consumption. This technique for improving the system of Zhang was within the ordinary ability of one of ordinary skill in the art based on the teachings of Srivastava.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang and Srivastava to obtain the invention as specified in claim 2.
Regarding claim 3, Zhang in view of Srivastava teaches the electronic system of claim 2, and Zhang further teaches that the first quality imagery comprises engine-quality imagery (e.g., [0011], the raw2 data output by preprocessing unit 230 is more suitable for AI/engine processing), and wherein the second quality imagery comprises human-quality imagery (This limitation substantially duplicates a requirement in claim 2 – see rejection of claim 2 for mapping).
Regarding claim 4, Zhang in view of Srivastava teaches the electronic system of claim 2, and Zhang further teaches that, to generate the first quality imagery, the edge PLD is further configured to:
convert a color fidelity, a dynamic range, and/or a compression state of the raw imagery to a lower color fidelity, a narrower dynamic range, and/or a relatively lossy compressed state relative to the raw imagery and/or a second quality processed version of the raw imagery (e.g., [0028]-[0031], Fig. 5, channel selection combiner 450 uses only a subset of color information – i.e., only Green, Blue, Red, or Blue + Red – to generate the first/engine/raw2 quality imagery, which has lower color fidelity than the full RGB provided by the second quality imagery produced by the ISP and the original raw imagery; Note that Fig. 3 illustrates a raw image at 300 and an RGB image at 310 and both include information from all three colors; Therefore, the first quality imagery has at least a lower color fidelity and/or a relatively lossy compressed state relative to the raw and/or second quality imagery);
applying engine-quality histogram equalization to the raw imagery;
applying engine-quality color correction to the raw imagery; and/or
applying engine-quality exposure control to the raw imagery.
Regarding claim 5, Zhang in view of Srivastava teaches the electronic system of claim 2, and Zhang further teaches that:
the one or more image tags comprises an object presence tag (e.g., [0021], the AI may perform object detection; The output of object detection falls within the scope of at least an object presence tag), an object bounding box tag, and/or one or more object feature status tags.
Examiner notes that Srivastava also teaches at least an object presence tag (e.g., [0022], face detection signal).
Regarding claim 8, Zhang in view of Srivastava teaches the electronic system of claim 2, and Srivastava further teaches that the controller is powered based, at least in part, on the one or more image tags and/or the generated second quality imagery (Note the rejection of claim 2 under 35 U.S.C. 112(b), which also applies to further dependent claim 8; See mapping provided in rejection of claim 2).
Regarding claim 9, Zhang in view of Srivastava teaches the electronic system of claim 8, and Zhang further teaches that the controller is configured to:
generate tagged second quality imagery corresponding to the received raw imagery based, at least in part, on the generated second quality imagery and the one or more image tags provided by the edge PLD (e.g., Fig. 2, [0022], output from AI 240 [i.e., the tags] is output as enhanced/additional information to display 220 of human-quality imagery, thus forming tagged human-quality imagery), and
display the tagged second quality imagery via a display of the electronic system ([0022], Fig. 2, display 220) and/or storing the tagged second quality imagery according to the one or more image tags associated with the second quality imagery.
Regarding claim 10, Zhang in view of Srivastava teaches the electronic system of claim 2, and Srivastava further teaches that the controller is configured to:
receive the one or more image tags and/or the generated first quality imagery from the edge PLD (e.g., [0022], face detection signal [i.e., a tag] is received by controller); and
generate a system response based, at least in part, on the one or more image tags and/or the generated first quality imagery, wherein the generating the system response comprises generating a user input (e.g., [0020], presentation of a login prompt), generating a system alert, disabling a display of the electronic system, and/or depowering the electronic system.
Regarding claim 13, Examiner notes that the claim recites a method that is substantially the same as the method performed by the system of claim 2. Zhang in view of Srivastava teaches the system of claim 2 (see above). Accordingly, claim 13 is also rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Srivastava for substantially the same reasons as claim 2.
Regarding claim 14, Zhang in view of Srivastava teaches the method of claim 13, and Zhang further teaches that the first quality imagery comprises one or more of a lower resolution, a lower color fidelity, a narrower dynamic range, and/or a relatively lossy compressed state relative to the raw imagery and/or a second quality processed version of the raw imagery (e.g., [0028]-[0031], Fig. 5, channel selection combiner 450 uses only a subset of color information – i.e., only Green, Blue, Red, or Blue + Red – to generate the first/engine/raw2 quality imagery, which has lower color fidelity than the full RGB provided by the second quality imagery produced by the ISP and the original raw imagery; Note that Fig. 3 illustrates a raw image at 300 and an RGB image at 310 and both include information from all three colors; Therefore, the first quality imagery has at least a lower color fidelity and/or a relatively lossy compressed state relative to the raw and/or second quality imagery).
Regarding claim 17, Examiner notes that the claim recites a method that is substantially the same as the method performed by the system of claim 8. Zhang in view of Srivastava teaches the system of claim 8 (see above). Accordingly, claim 17 is also rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Srivastava for substantially the same reasons as claim 8.
Regarding claim 19, Zhang in view of Srivastava teaches the method of claim 13 (see above). Examiner notes that claim 19 recites a limitation that is substantially repeated from claim 12 – see the rejection under 35 U.S.C. 112(d). Zhang further teaches this limitation – see rejections of claims 12 and 1 under 35 U.S.C. 103. Therefore, claim 19 is also rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Srivastava.
Regarding claim 20, Zhang in view of Srivastava teaches the method of claim 19, and Zhang further teaches:
generating tagged second quality imagery corresponding to the received raw imagery based, at least in part, on the generated second quality imagery and the one or more image tags provided by the edge PLD (e.g., Fig. 2, [0022], output from AI 240 [i.e., the tags] is output as enhanced/additional information to display 220 of human-quality imagery, thus forming tagged human-quality imagery), and
displaying the tagged second quality imagery via a display of the electronic system ([0022], Fig. 2, display 220) and/or storing the tagged second quality imagery according to the one or more image tags associated with the second quality imagery; and/or
generating a system alert, disabling the imaging module of the electronic system, disabling the display of the electronic system, and/or depowering the electronic system.
Regarding claim 21, Examiner notes that the claim recites a method that is substantially the same as the method performed by the system of claim 10. Zhang in view of Srivastava teaches the system of claim 10 (see above). Accordingly, claim 21 is also rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Srivastava for substantially the same reasons as claim 10.
Claim(s) 11 and 22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Srivastava as applied above, and further in view of ‘Buckler’ (“Reconfiguring the Imaging Pipeline for Computer Vision,” 2017; cited and copy provided in “parent” application 18/464,175).
Regarding claim 11, Zhang in view of Srivastava teaches the electronic system of claim 2.
Zhang further teaches that the image engine of the edge PLD is implemented as a neural network, a machine learning, and/or an artificial intelligence-based image processing engine (Fig. 2, artificial intelligence (AI) 240).
Zhang does not teach details regarding how the image engine is trained. In particular, Zhang does not explicitly teach training by:
generating a first quality training set of training images and associated image tagging based, at least in part, on a second quality training set of training images and associated image tagging corresponding to a desired selection of image tags; and
determining a set of weights for the image engine based, at least in part, on the first quality training set.
Srivastava also does not explicitly teach these features.
However, Buckler does teach training an image engine to generate one or more image tags (e.g., Section 3.2 and Figs. 1(b) and 2) by:
generating a first quality training set of training images (Fig. 2, converted dataset) and associated image tagging (e.g., Sec. 3.2, 1st paragraph, the images in the converted dataset are labeled) based, at least in part, on a second quality training set of training images (Fig. 2, original dataset) and associated image tagging corresponding to a desired selection of image tags (e.g., Sec. 3.2, 1st par., existing dataset is labeled; e.g., Table 1, for object classification, CIFAR-10 dataset is used, which tags each image from a selection of desired/possible image object type tags); and
determining a set of weights for the image engine based, at least in part, on the first quality training set (e.g., Sec. 3.3, last par., CNN image engine is trained – i.e., an optimal set of weights is determined – using the generated engine-quality training set).
Any artificial intelligence (e.g., convolutional neural network (CNN)) model requires training. Training requires a dataset. Accordingly, one seeking to implement the electronic system of Zhang would have to obtain a training dataset and use it to train the AI model.
Buckler uses a similar approach where first/engine-quality imagery is input to an AI model (e.g., Fig. 1(a)) and thus requires a training dataset that can be used to train its AI model (e.g., Sec. 3.2, 1st par.). One approach for obtaining a training dataset would be to generate one from scratch by capturing first/engine-quality imagery and labeling it (e.g., Sec. 3.2, 1st par.), but this would be time-consuming and expensive. Buckler proposes to instead convert and existing, second/human-quality training image dataset into a first/engine-quality training image dataset (e.g., Sec. 3.2, 1st and 2nd pars.). This advantageously allows a first/engine-quality training image dataset to be generated in only one hour (e.g., Sec. 3.2, last par.).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the system of Zhang in view of Srivastava with the dataset generation and training of Buckler in order to improve the system with the reasonable expectation that this would result in a system that could train its AI model using a dataset obtained in an efficient manner. This technique for improving the system of Zhang in view of Srivastava was within the ordinary ability of one of ordinary skill in the art based on the teachings of Buckler.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang, Srivastava and Buckler to obtain the invention as specified in claim 11.
Regarding claim 22, Examiner notes that the claim recites a method that is substantially the same as the method performed by the system of claim 11. Zhang in view of Srivastava and Buckler teaches the system of claim 11 (see above). Accordingly, claim 22 is also rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Srivastava and Buckler for substantially the same reasons as claim 11.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Zhang in view of Srivastava as applied above, and further in view of ‘Caviedes’ (US 2016/0173752 A1).
Regarding claim 18, Zhang in view of Srivastava teaches the method of claim 13.
Both Zhang (e.g., [0037]) and Srivastava (e.g., [0001], [0020]) are concerned with power consumption, but neither explicitly teaches:
monitoring a charge state of a power supply of the electronic system; and
controlling a frame rate of the imaging module based, at least in part, on the monitored charge state of the power supply.
However, Caviedes does teach:
monitoring a charge state of a power supply of the electronic system ([0051], battery charge is monitored); and
controlling a frame rate of the imaging module based, at least in part, on the monitored charge state of the power supply ([0051], power configuration is changed – e.g., to medium – if threshold charge state is reached; [0051], power configuration – e.g., medium power configuration – controls frame rate of imaging module).
Caviedes teaches that such power control advantageously ensures that a computer vision process can operate "without exceeding, or substantially depleting available device power" and in a manner that prolongs battery life ([0051]).
Before the effective filing date of the claimed invention, it would have been obvious to one of ordinary skill in the art to modify the method of Zhang in view of Srivastava as applied above with the power configuration control of Caviedes in order to improve the method with the reasonable expectation that this would result in a method that could further manage power consumption in a manner that advantageously ensured its computer vision processes could operate without exceeding, or substantially depleting available device power and in a manner that prolongs battery life. This technique for improving the method of Zhang in view of Srivastava was within the ordinary ability of one of ordinary skill in the art based on the teachings of Caviedes.
Therefore, it would have been obvious to one of ordinary skill in the art to combine the teachings of Zhang, Srivastava, and Caviedes to obtain the invention as specified in claim 18.
Claims Not Rejected Over Prior Art
Examiner notes that claims 6-7 and 15-16 are not rejected over prior art. However, these claims are rejected under 35 U.S.C. § 112(a) and § 112(b) so they are not in condition for allowance.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GEOFFREY E SUMMERS whose telephone number is (571)272-9915. The examiner can normally be reached Monday-Friday, 7:00 AM to 3:30 PM ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chan Park can be reached at (571) 272-7409. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GEOFFREY E SUMMERS/Examiner, Art Unit 2669
Prosecution Insights