DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claims 1 and 8 are amended. Claims 3-4 and 9-11 are cancelled. Claims 1-2, 5-8 are pending. New matter is added.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114.
Response to Arguments
Applicant's arguments filed have been fully considered but they are not persuasive.
The amendment filed is objected to under 35 U.S.C. 132(a) because it introduces new matter into the disclosure. 35 U.S.C. 132(a) states that no amendment shall introduce new matter into the disclosure of the invention. The added material which is not supported by the original disclosure is as follows:
Applicant is required to cancel the new matter in the reply to this Office Action.
Independent claims 1 and 8 recite similar limitations – “A red, green, blue and infrared (RGBIR) image processing system, comprising: an RGBIR-Bayer converter that includes image processing circuitry configured to convert RGBIR raw data and exclude[[s]] infrared (IR) raw data to result in Bayer raw data without JR raw data; and a Bayer-RGB converter that includes image processing circuitry configured to convert the Bayer raw data to RGB color data; wherein the RGBIR-Bayer converter comprises a subsampler that includes subsampling circuitry configured to…”, wherein the processing circuitry appears to be the hardware structure to execute functions recited by RGBIR Bayer converter and subsampler.
However, the original specifications do not explicitly processing circuitry and therefore new matter is introduced in the amended claims as filed. Further, the Applicant remarks that the fig. 1 discloses hardware processor structure in processing system. However, the specifications do not explicitly link the hardware processing structure of element#100 to any of Bayer RGBIR converter or subsampler directly.
Further, the original specifications recite -- “A co-processor is a device that works alongside a main system-on-chip (SoC) in a smart camera system. The co-processor handles the computation of image analytics, which requires low resolution images. The main SoC, on the other hand, processes high resolution images for human viewing or advanced analytics. This way, the co-processor reduces the workload of the main SoC and improves the performance of the smart camera system. One of the benefits of using the co-processor is that it enables constant sensing without draining the battery of the system. The co-processor can detect relevant events and activate the SoC only when needed. This way, the system can save power and extend its operation time. For example, a home surveillance system can use the co-processor to monitor the presence of humans in the area. When the co-processor senses a human, it can trigger the SoC to start recording video. This reduces the amount of unnecessary video data and power consumption of the system”, and “FIG. 1 shows a block diagram illustrating a red, green, blue and infrared (RGBIR) image processing system 100 according to one embodiment of the present invention. The RGBIR image processing system 100 of FIG. 1 may be adapted to an always-on co-processor (e.g., image signal processor or ISP) in a smart camera system to wake a system-on-chip (SoC) only when a relevant event occurs. In the embodiment, the RGBIR image processing system 100 may include an RGBIR-Bayer converter 11 configured to convert RGBIR raw data to Bayer raw data. Specifically, the RGBIR raw data is an image data captured by an (RGBIR) image sensor covered with an RGBIR filter array having an RGBIR pattern, as exemplified in the 4x4 pattern in FIG. 1, which is half green, one quarter infrared, one eighth red and one eighth blue. The Bayer raw data is an image data arranged in Bayer pattern, as exemplified in the 4x4 pattern in FIG. 1, which is half green, one quarter red and one quarter blue” in detailed descriptions. However, the claimed and underlined fig. 1 as hardware structure of image processing system 100 may be adapted to an always on co-processor in smart camera system to wake a system on chip. Therefore, the feature of adapted to is open-ended and does not restrict the only use of hardware structure and may comprise of any other software modalities to execute the functions and therefore, the interpretations and rejections are maintained.
Examiner suggests amending claims and specifications to explicitly define the hardware processors as recited in claims that specifically corelates to each particular Bayer RGBIR converter and Subsampler in order to remove rejections and interpretations as below.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier.
Claims 1-2 and 5-8 recites limitations that use words like “means” (or “step”) or similar terms with functional language and do invoke 35 U.S.C. 112(f):
• Claim 1; recites the limitation, “A RGBIR converter [Line 2].
• Claim 1; recites the limitation, “a Bayer RGB converter.” [Line 4].
Claim 8; recites the limitations, “an RGBIR Bayer converter” [Lines 1].
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
After a careful analysis, as disclosed above, and a careful review of the specification the following limitations in claims 1 and 8:
“a RGBIR converter” (page 5) - RGBIR image processing system 100may include an RGBIR-Bayer converter 11 configured to convert RGBIR raw data to Bayer raw data. Specifically, the RGBIR raw data is an image data captured by an (RGBIR) image sensor covered with an RGBIR filter array having an RGBIR pattern, as exemplified in the 4x4 pattern in FIG. 1,which is half green, one quarter infrared, one eighth red and one eighth blue. The Bayer raw data is an image data arranged in Bayer pattern, as exemplified in the 4x4 pattern in FIG. 1, which is half green, one quarter red and one quarter blue. Fig. 1, illustrates the RGBIR converter as a black box. (wherein the RGBIR converter do not have sufficient structure associated with it.).
“a Bayer RGB converter” (page 7]- a Bayer-RGB converter 12 configured to convert the Bayer raw data to RGB color data (or image). According to another aspect of the embodiment, the Bayer-RGB converter 12may include a color adjusting device 121 configured to adjust color information of the Bayer raw data, thereby generating adjusted data. In one embodiment, the color adjusting device 121 may perform color processing and tone processing on the Bayer raw data. The color processing may, for example, include white balance and color correction. White balance is a process of adjusting colors (e.g., color temperature) in an image to make it appear more natural by removing color casts caused by the lighting conditions. Color correction, on the other hand, is a process of adjusting colors in an image to achieve a desired look or mood. The tone processing may, for example, include gamma correction. Gamma correction is a process of adjusting brightness of an image, and is commonly used to correct the brightness of images that appear too dark or too bright. For per-pixel process likewhite balance and gamma correction, the computation is performed individually for each pixel based on its color channel. For process requiring RGB information like color correction, the processing is done for each 2x2 Bayer pattern. FIG. 4 shows an exemplary 2x2 Bayer pattern subjected to color correction. In the embodiment, color correction is executed twice for every 2x2 Bayer pattern, once for each green pixel G1 or G2. The output red and blue pixel values are then derived by averaging the two corrected red values (R1 and R2) and the two blue pixel values (B1 and B2).. Fig. 1, illustrates the Bayer RGB converter as a black box. (wherein the Bayer RGB converter do not have sufficient structure associated with it.).
“a subsampler” (page 6]- the RGBIR-Bayer converter 11 may include a subsampler 111 configured to subsample the RGBIR raw data to generate subsampled data, which preserves the RGBIR pattern. According to one aspect of the embodiment, an odd-numbered subsampling scheme is adopted. Specifically, the subsampler 111 decimates the RGBIR raw data by odd number M (e.g., 3, 5 or 7) to keep only every RGBIR raw data at M-th row and M-th column. Equivalently speaking, only every RGBIR raw data in the upper left corner of MxM matrix is kept as the subsampled data. FIG. 2 shows exemplary subsampled data decimated by the subsampler III of FIG. 1 adopting the odd-numbered subsampling scheme with odd number 3 to keep only every RGBIR raw data at third row and third column, thereby preserving the RGBIR pattern. As shown in FIG. 2, adjacent subsampled data are regularly spaced. FIG. 3 shows exemplary subsampled data not adopting the odd-numbered subsampling scheme of the embodiment. As shown in FIG. 3, adjacent subsampled data are irregularly spaced. . Fig. 1, illustrates an subsampler as a black box. (wherein the subsampler do not have sufficient structure associated with it.).
“a demosaicing device” (page 8]- The Bayer-RGB converter 12 of the embodiment may include a demosaicing device 122 configured to convert the adjusted data to the RGB color data in order to reconstruct a full color image. Conventional dernosaicing techniques may be adopted, detail of which may, for example, be referred to US Patent No. 11,836,888, contents of which are incorporated herein by reference. Fig. 1, illustrates the demosaicing device as a black box. (wherein the demosaicing device do not have sufficient structure associated with it.).
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 along with their dependent claims 2, 5-7, are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
• Claims 1 and 8 recites limitations:
• Claim 1; recites the limitation, “a RGBIR converter.” [Line 2].
• Claim 1; recites the limitation, “a Bayer RGB converter [Line 4].
Claim 8; recites the limitations, “an RGBIR Bayer converter” [Lines 1].
Claims 1 and 8 respectively invokes 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. However, the written description fails to disclose the corresponding structure, material, or acts for performing the entire claimed function and to clearly link the structure, material, or acts to the function. The specification is devoid of adequate structure to perform the claimed functions. The specification does not provide sufficient details such that one of the ordinary skill in the art would understand which structure performed(s) the claimed function.
Therefore, the claim is indefinite and is rejected under 35 U.S.C. 112(b) or pre-AIA 35 U.S.C. 112, second paragraph.
Applicant may:
(a) Amend the claim so that the claim limitation will no longer be interpreted as a limitation under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph;
(b) Amend the written description of the specification such that it expressly recites what structure, material, or acts perform the entire claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(c) Amend the written description of the specification such that it clearly links the structure, material, or acts disclosed therein to the function recited in the claim, without introducing any new matter (35 U.S.C. 132(a)).
If applicant is of the opinion that the written description of the specification already implicitly or inherently discloses the corresponding structure, material, or acts and clearly links them to the function so that one of ordinary skill in the art would recognize what structure, material, or acts perform the claimed function, applicant should clarify the record by either:
(a) Amending the written description of the specification such that it expressly recites the corresponding structure, material, or acts for performing the claimed function and clearly links or associates the structure, material, or acts to the claimed function, without introducing any new matter (35 U.S.C. 132(a)); or
(b) Stating on the record what the corresponding structure, material, or acts, which are implicitly or inherently set forth in the written description of the specification, perform the claimed function. For more information, see 37 CFR 1.75(d) and MPEP §§ 608.01(o) and 2181.
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
• Claim 1 along with their dependent claims 2-11, are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for pre-AIA the inventor(s), at the time the application was filed, had possession of the claimed invention. As described above, the disclosure does not provide adequate structure to perform the claimed function in the recited limitation Claims 1-11 recites limitations.
• Claim 1; recites the limitation, “a RGBIR converter [Line 2].
• Claim 1; recites the limitation, “a Bayer RGB converter.” [Line 4].
The specification does not demonstrate that applicant has made an invention that achieves the claimed function because the invention is not described with sufficient detail such that one of ordinary skill in the art can reasonably conclude that the inventor had possession of the claimed invention.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1 and 8 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 4 recite limitations – “wherein the subsampler decimates the RGBIR raw data by odd number M to keep only every RGBIR raw data at M-th row and M-th column”, appears to be directed to only retain RGBIR raw data at Mth row and Mth column. The M is not explicitly specified being which specific odd number, therefore, it is not clear as to which of the M row data and column data is retained and if each odd numbered Mth row data and column data or only one selected odd M data is subsampled.
Therefore, Examiner suggests to modify the data in order to explicitly define the M number to render the claims definite.
Claim 8 recite limitations – “wherein the color correction is executed twice for every 2x2 Bayer pattern, once for each green pixel, thereby generating two corrected red values and two blue pixel values; and the output red and blue pixel values are then derived by averaging the two corrected red values and the two blue pixel values”, appears to be directed to color correction is executed two times for each green and blue pixels. However, the claimed features only recite once for every green pixel and not reciting every red pixel.
Therefore, Examiner suggests amending the claims to explicitly define the color correction process applied to bayer pattern for green and blue pixels in order to render the claims definite.
Allowable Subject Matter
Claims 1-2 and 5-8 would be allowable if rewritten to overcome the rejection(s) under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), 2nd paragraph 35 U.S.C. 112(a), set forth in this Office action by amending claims to remove new matter or amending specifications to include the recited new matter into a CON application.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
US-20200112662-A1 (Sakamoto et al., The present technology relates to an imaging device, an image processing method, and an image processing system which are configured to obtain a sensing image and a viewing image by one imaging element. The imaging device includes: a control unit that controls irradiation with invisible light; an imaging element that includes a first pixel that is capable of detecting the invisible light, and a second pixel that is capable of detecting visible light; and an image generation unit that generates a first image including the invisible light component and a second image including a visible light component on the basis of a first pixel signal transmitted from the first pixel and a second pixel signal transmitted from the second pixel. For example, the present technology is applicable to a monitoring camera, Abstract).
US-20220182564-A1 (An image processing system includes: an image sensor suitable for generating an RYYB (Red Yellow Yellow Blue) bayer image by applying an RYYB color filter array; an interpolation logic suitable for generating a Y image at a position of an R image portion in the RYYB bayer image by interpolating a YY image portion in the RYYB bayer image; and a guided filtering logic suitable for guided-filtering the R image portion in the RYYB bayer image by using, as a guide image, the Y image at the position of the R image portion, Abstract)
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PINALBEN V PATEL whose telephone number is (571)270-5872. The examiner can normally be reached M-F: 10am - 8pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Wills-Burns Chineyere can be reached at 571-272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Pinalben Patel/Examiner, Art Unit 2673
Prosecution Insights