Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 05/24/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Drawings
The drawings are objected to because of the following:
Figures 3, 5, 6, and 7 all appear to be identical as simply a gray box. It is recommended that labels be added that depict the areas in each figure that have specific defective pixels.
Figures 10 and 11 both fail to include reference labels that point to the specific elements depicted.
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Specification
The disclosure is objected to because of the following informalities:
On page 9, paragraph [0081], penultimate line - “Figure 2” should be changed to --Image 2--. Appropriate correction is required.
Claim Objections
Claim 7 is objected to because of the following informalities: in claim 7, line 5, “a second” should be changed to --the second--. Appropriate correction is required.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are:
In claim 8, all nine of the “modules” set forth.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. The claim(s) does/do not fall within at least one of the four categories of patent eligible subject matter because claim 9 sets forth a “computer-readable storage medium” that has not been limited to non-transitory embodiments only. The specification at [0160] describes in an open-ended manner the various forms that a computer-readable storage medium may take. Due to the failure of the specification to limit the claimed CRSM to non-transitory embodiments only, the claim breadth also encompasses signals per se and transitory media such as copper wire and optical fiber.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-5, 7-9 are rejected under 35 U.S.C. 103 as being unpatentable over CN112734719A, hereinafter “CN’719” in view of US 2017/0160127 to Salisbury et al. Note that all citations to CN’719 are based upon the supplied English translation of CN’719.
As per claim 1, CN’719 discloses on page 3 a defective pixel correction method, characterized by comprising: step A, acquiring an initial image generated by an infrared imaging device if the infrared imaging device is detected to meet a preset trigger condition (page 3 - acquiring a first image via a CCD or CMOS imaging sensor but CN’719 fails to disclose the triggering condition for IR capture); step B, determining whether each point in the initial image is a possible defective pixel according to a preset defective pixel determination feature (page 3, obtaining the contrast of the N units of the first image and storing in the first memory while performing suspected bad pixel detection on the N units of the first image to obtain a set of first image suspected pixel coordinates and subsequently storing these in a second memory); step C, summarizing all the possible defective pixels in the initial image to obtain a first possible defective pixel table (page 3, first pixel table (coordinates) stored in the second memory); step D, for two consecutive image frames generated in real-time by the infrared imaging device, determining the overlapping area of the two consecutive image frames (page 3, acquiring a second image being taken after the first image and comparing pixel locations of the first image with corresponding pixel locations (overlap) of the second image); step E, determining the possible defective pixels located in the overlapping area in a latter frame of the two consecutive image frames (page 3, take absolute value of the difference between the contrast values in the first memory vs the second image), and form a second possible defective pixel table (page 3, if absolute value is greater than a threshold); step F, determining a non-repetition part of the possible defective pixels in the overlapping area in the latter image frame in the current first possible defective pixel table and the second possible defective pixel table (page 3, determining the intersection of the coordinates of the suspected bad pixels in the second image corresponding to the suspected defective pixels in the second memory); step G, after the non-repetition part is eliminated from the first possible defective pixel table (page 3, per the threshold, the contrast values of the first memory are replaced (i.e. eliminated) with the contrast of the second image), obtaining an updated first possible defective pixel table; step H, repeating step D - step G to obtain a final first possible defective pixel table (page 3, the images subsequent to the second image are sequentially detected to obtain the coordinate set of the dead pixels); and step I, performing defective pixel correction on the basis of the final first possible defective pixel table (bottom of page 3, performing a repair operation).
As noted above, CN’719 fails to disclose an IR sensor where motion of the IR sensor is monitored, and motion of the IR sensor beyond a threshold triggers the capture of the first image and subsequent pixel detection method.
However, in the same field of endeavor as CN’719, Salisbury et al discloses a defective pixel detection method for IR sensors that monitors the motion of the IR sensor and triggers the defective pixel algorithm if a motion threshold is exceeded, see Salisbury et al, [0006] - [0012].
Therefore, it would have been obvious before the effective filing date of the claimed invention to have provided the image sensor of CN’719 with IR capability while also initiating the defective pixel evaluation only after motion of the sensor exceeds a threshold as taught by Salisbury et al. The rationale being that an IR sensor permits capture of images in very low light situations which is very challenging for visible light sensors, while the motion monitoring of the sensor and only permitting defective pixel evaluation after a motion threshold has been exceeded reduces the amount of processing resources and obtains defective pixel locations more rapidly, see Salisbury et al [0007].
As per claim 2, CN’719 in view of Salisbury et al discloses the method according to claim 1, characterized by further comprising: acquiring a motion characteristic of the infrared imaging device in real-time; and confirming that the infrared imaging device meets a preset trigger condition when the motion amplitude of the infrared imaging device is determined to be greater than a preset amplitude threshold on the basis of the motion characteristics (Salisbury et al [0013]).
As per claim 3, CN’719 in view of Salisbury et al discloses the method according to claim 2, characterized in that acquiring the motion characteristics of the infrared imaging device in real-time comprises: real-time monitoring of the infrared imaging device using a motion situation sensor to obtain the motion characteristics of the infrared imaging device (Salisbury et al [0010].
As per claim 4, CN’719 in view of Salisbury et al discloses the method according to claim 1, characterized in that the defective pixel determination feature is generated on the basis of the difference between the defective pixel and a normal pixel (see CN’719, page 1, potential defective pixels are determined by comparing the pixel value to an average of pixels surrounding said pixel wherein the surrounding pixels are ‘normal’ pixels).
As per claim 5, CN’719 in view of Salisbury et al discloses the method according to claim 1, characterized in that the number of repetitions of step D - step G is related to the accuracy requirement; wherein the higher the accuracy requirement, the more corresponding times (see CN’719, page 2 where the defective pixel steps of comparing frames is repeated on other frames in order to obtain the real dead pixel coordinate set. It stands to follow that one of ordinary skill in the art would have known that the more frames that are compared, the more the final set of dead pixel coordinates can be fine-tuned with increased accuracy of the coordinates).
As per claim 7, CN’719 in view of Salisbury et al discloses the method according to claim 1, characterized in that the step E comprises: determining whether each point in the overlapping area in the latter frame of the two consecutive image frames is a possible defective pixel based on the preset defective pixel determination feature; and summarizing all the possible defective pixels in the overlapping area to obtain a second possible defective pixel table (see CN’719 at page 3, the absolute value of the difference between first and second images (N units) is compared to a threshold).
Claim 8 is rejected for reasoning, mutatis mutandis, as that of claim 1 above.
As per claim 9, CN’719 discloses the claimed computer-readable storage medium at page 1, under “Summary of the Invention” (“storage medium”).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over CN’719 in view of Salisbury et al as applied to claims 1-5, 7-9 above, and further in view of WO2010069878A1, hereinafter “WO’878.” Note that citations below directed to WO’878 are based upon the supplied English translation of WO’878.
CN’719 in view of Salisbury et al teach the overlapping of two frames of images, but fail to teach that for two consecutive frames of images generated in real-time by the infrared imaging device, separately calculating the projection of the two consecutive image frames on the X direction and the projection in a preset rectangular coordinate system; determining a translation amount of the two consecutive image frames in the X direction and the Y direction based on the projection of the two consecutive image frames in the X direction and the Y direction; based on the translation amount of the two consecutive image frames in the X direction and the translation amount in the Y direction, determining the overlapping area of the two consecutive image frames.
However, the in the same field of endeavor, WO’878 teaches at [0028] a defective pixel method that, as illustrated in figures 2, performs overlapping of two image frames by realigning two images with respect to each other from the determined offset D (step B), in order to be able to compare the pixels representing the same information of the visualized scene: a realigned reference image 2' is obtained. Pixel locations are generally determined using an orthonormal Ox, Oy reference frame. Image registration then consists of performing a translation of the reference frame.
Therefore, it would have been obvious before the effective filing date of the claimed invention to have performed the overlapping of image frames in CN’719 in view of Salisbury et al via the translation method taught by WO”878 as doing this would provide for accurate and repeatable overlapping of frame pairs by determining the corresponding special alignment of the individual pixels between the frame pairs.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. The prior art cited is emblematic of the state of the art in defective pixel detection/correction where multiple frames of images are considered when deciding which pixels of the sensor array are defective.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DAVID OMETZ whose telephone number is (571)272-7593. The examiner can normally be reached M-F, 8am-4pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Sumati Lefkowitz can be reached at 571-272-3638. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DAVID OMETZ
Primary Examiner
Art Unit 2672
/DAVID OMETZ/Primary Examiner, Art Unit 2672