Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Election/Restrictions
Applicant’s election without traverse of claims 1-13 in the reply filed on 04/22/2026 is acknowledged.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 2, 7, 8, 13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20130033585A1; February 7, 2013) in view of Chatterjee (US 2018/0122061A1; May 3, 2018).
Regarding claim 1, Li teaches a method of performing color calibration of a image sensor, the method comprising:
obtaining test measurement data of at least one color chart that is measured by a test sensor under at least one lighting environment (Paragraph 23 camera is subjected to color matching calibration with color targets, such as Macbeth ColorChecker chart, and using a standard illuminant reference and observer. Paragraph 26. The raw color responses of the golden camera and the camera to be calibrated may be designated as rgb.sub.24.times.3 and rgb'.sub.24.times.3. rgb.sub.24.times.3 and rgb'.sub.24.times.3 may be obtained in calibration image blocks (e.g., 246, 248, and 247). Paragraph 35, an illuminant in the 4500K CCT range, to conduct color matching);
obtaining reference measurement data of the at least one color chart that is measured by a reference sensor under the at least one lighting environment, the reference sensor being calibrated in advance (Paragraph 23, 26, The raw color responses of the golden camera and the camera to be calibrated may be designated as rgb.sub.24.times.3 and rgb'.sub.24.times.3. rgb.sub.24.times.3 and rgb'.sub.24.times.3 may be obtained in calibration image blocks (e.g., 246, 248, and 247). Paragraph 35) the reference sensor being calibrated in advance; and (Paragraph 23 the golden camera is subjected to color matching calibration with color targets, such as Macbeth ColorChecker chart, and using a standard illuminant reference and observer).; and
generating, based on the test measurement data and the reference measurement data, at least one transformation model configured to transform measurements between the test sensor and the reference sensor (Paragraph 26, each camera in the system may be calibrated with the golden camera for color matching to determine the color matching matrix M'3x3 for that camera. The raw color responses of the golden camera and the camera to be calibrated may be designated as rgb24x3 and rgb'24x3 rgb24x3 and rgb'24x3 may be obtained in calibration image blocks (e.g., 246, 248, and 247). Preferably, M3x3 has been applied in color correction block). Paragraphs 27, the color matching matrix between the golden camera and the camera to be calibrated, M'3x3, may be obtained according to the following equation: rgb24x3=rgb'24x3M'3x3. Paragraph 29 Application of the color correction matrix (M) to the master and slave channels 110, 120 occurs in color correction steps 242 and 244. Color matching between the master and slave channels 110, 120 includes integration of the color correction and color matching matrices. Paragraph 35). Li generates a matrix M’ which transforms the measurements of the camera to be calibrated into corresponding measurements of the golden camera.
Li does not teach a multispectral image sensor (MIS).
Chatterjee teaches a multispectral image sensor (MIS) (Paragraph 37 -the cameras can be multispectral Paragraph 33)
Therefore, from the teaching of Chatterjee, it would have been obvious at the time of filing to specify the abovementioned limitation in order allow for color calibration of a multispectral image that would yield an MIS with increased detection accuracy.
Regarding claim 2, Li in view of Chatterjee teach the method of claim 1. Li further teaches wherein the test measurement data comprises a test measurement data matrix comprising rows corresponding to channels of the test sensor and columns corresponding to color samples in the at least one color chart (Paragraphs 26-28), and
wherein the reference measurement data comprises a reference measurement data matrix comprising rows corresponding to channels of the reference sensor and columns corresponding to the color samples of the at least one color chart (Paragraph 23, 28).
Li does not teach a multispectral image sensor (MIS).
Chatterjee teaches a multispectral image sensor (MIS) (Paragraph 37 -the cameras can be multispectral Paragraph 33)
Therefore, from the teaching of Chatterjee, it would have been obvious at the time of filing to specify the abovementioned limitation in order allow for color calibration of a multispectral image that would yield an MIS with increased detection accuracy.
Regarding claim 7, Li in view of Chatterjee teach the method of claim 2. Li further teaches wherein the at least one lighting environment comprises a first lighting environment and a second lighting environment (Paragraph 23, In steps 238 and 240, the golden camera is subjected to color matching calibration with color targets, such as Macbeth ColorChecker chart, and using a standard illuminant reference and observer. An exemplary calibration includes the CIE 1931 standard colorimetric observer at selected illuminants, including illuminant A and D65.)
wherein the obtaining of the test measurement data comprises:
obtaining first test measurement data of the at least one color chart that is measured by the test sensor under the first lighting environment illuminated with a first illuminant (Paragraph 23, 26, 28);
obtaining second test measurement data of the at least one color chart that is measured by the test sensor under the second lighting environment illuminated with a second illuminant (Paragraph 23, 26, 28); and
generating the test measurement data matrix based on the first test measurement data and the second test measurement data (Paragraph 23, 26, 28, 31), and
wherein the obtaining of the reference measurement data comprises:
obtaining first reference measurement data of the at least one color chart that is measured by the reference sensor under the first lighting environment illuminated with the first illuminant; obtaining second reference measurement data of the at least one color chart that is measured by the reference sensor under the second lighting environment illuminated with the second illuminant (Paragraph 23, 26, 28, 31); and
generating the reference measurement data matrix based on the first reference measurement data and the second reference measurement data (Paragraph 23, 26, 28, 31).
Li does not teach a multispectral image sensor (MIS).
Chatterjee teaches a multispectral image sensor (MIS) (Paragraph 37 -the cameras can be multispectral Paragraph 33)
Therefore, from the teaching of Chatterjee, it would have been obvious at the time of filing to specify the abovementioned limitation in order allow for color calibration of a multispectral image that would yield an MIS with increased detection accuracy.
Regarding claim 8, Li in view of Chatterjee teach the method of claim 7. Li further teaches wherein the first illuminant is different from the second illuminant (Paragraph 23 – selected illuminants, including illuminant A and D65).
Regarding claim 13, Li in view of Chatterjee teach the method of claim 7. Li further teaches comprising: transforming measurement data measured by the test sensor using the at least one transformation model (Paragraph 23-24, 26-27, 29); and obtaining calibrated color data from the measurement data that is transformed using a reference color calibration model of the reference sensor (Paragraph 23-24, 26-27, 29).
Li does not teach a multispectral image sensor (MIS).
Chatterjee teaches a multispectral image sensor (MIS) (Paragraph 37 -the cameras can be multispectral Paragraph 33)
Therefore, from the teaching of Chatterjee, it would have been obvious at the time of filing to specify the abovementioned limitation in order allow for color calibration of a multispectral image that would yield an MIS with increased detection accuracy.
Claim(s) 3-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20130033585A1; February 7, 2013) in view of Chatterjee (US 2018/0122061A1; May 3, 2018) in view of Zarcone (US 2021/0298143 A1; September 23, 2021).
Regarding claim 3, Li in view of Chatterjee teaches the method of claim 2, but fails to teach wherein the generating of the at least one transformation model comprises: calculating the at least one transformation model by multiplying the reference measurement data matrix by an inverse matrix of the test measurement data matrix.
Zarcone teaches wherein the generating of the at least one transformation model comprises: calculating the at least one transformation model by multiplying the reference measurement data matrix by an inverse matrix of the test measurement data matrix. (Paragraph 5, 114 - A method for color correction includes computing a reference panel matrix (P.sub.R) based on colors (which can be represented as data) of a reference panel light; computing a target panel matrix (P.sub.T) based on light measured from a target panel; calculating an inverse matrix (P.sub.T.sup.−1) for the target panel matrix;)
Therefore, from the teaching of Zarcone, it would have been obvious at the time of filing to specify the abovementioned limitation since it is a known method of color correction.
Regarding claim 4, Li in view of Chatterjee in view of Zarcone teach the method of claim 3. Li further teaches wherein the at least one transformation model is an NxN matrix comprising column corresponding to N channels in the reference sensor and row corresponding to N channels of the test sensor (Paragraph 26-28).
Li doesn’t teach wherein the at least one transformation model is an NxN matrix comprising row corresponding to N channels in the reference sensor and column corresponding to N channels of the test sensor.
However, it would have been obvious at the time of filing to specify the abovementioned limitation since it is a simple matter of design choice in how to present the transformation model.
Li does not teach a multispectral image sensor (MIS).
Chatterjee teaches a multispectral image sensor (MIS) (Paragraph 37 -the cameras can be multispectral Paragraph 33)
Therefore, from the teaching of Chatterjee, it would have been obvious at the time of filing to specify the abovementioned limitation in order allow for color calibration of a multispectral image that would yield an MIS with increased detection accuracy.
Claim(s) 5 is rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20130033585A1; February 7, 2013) in view of Chatterjee (US 2018/0122061A1; May 3, 2018) in view of Day (A Psychophysical Experiment Evaluating the Color Accuracy of Several Multispectral Image Capture Techniques; January, 2003).
Regarding claim 5, Li in view of Chatterjee teaches the method of claim 2. Li further teaches obtaining measurement data … with respect to each of M color samples in the at least one color chart, and (Paragraph 26, The raw color responses of the golden camera and the camera to be calibrated may be designated as rgb.sub.24.times.3 and rgb'.sub.24.times.3. rgb.sub.24.times.3 and rgb'.sub.24.times.3. Paragraph 28, the R, G, and B values of the 24 color patches of the Macbeth Color Checker chart may be designated as rgb.sub.24.times.3)
wherein the test measurement data matrix is an Nx(M…) matrix comprising rows corresponding to N channels in the [sensor/MIS] and columns corresponding to the … M color samples of the at least one color chart (Paragraph 26, The raw color responses of the golden camera and the camera to be calibrated may be designated as rgb24x3 and rgb'24x3. Paragraph 28, the R, G, and B values of the 24 color patches of the Macbeth Color Checker chart may be designated as rgb.sub.24.times.3)
Li does not teach a multispectral image sensor (MIS).
Chatterjee teaches a multispectral image sensor (MIS) (Paragraph 37 -the cameras can be multispectral Paragraph 33)
Therefore, from the teaching of Chatterjee, it would have been obvious at the time of filing to specify the abovementioned limitation in order allow for color calibration of a multispectral image that would yield an MIS with increased detection accuracy.
Li in view of Chaterjee fail to teach obtaining measurement data of P pixels with respect to each of M color samples in the at least one color chart, and wherein the test measurement data matrix is an Nx(M*P) matrix comprising rows corresponding to N channels in the MIS and columns corresponding to the P pixels in each of the M color samples of the at least one color chart.
Day teaches obtaining measurement data of P pixels (Page 2, the number of pixels per patch …represented by p) with respect to each of M (Page 2, the number of patches are represented by p and n) color samples in the at least one color chart, and (Page 2 target included a Gretag Macbeth ColorChecker DC … All transformation matrices were created using the ColorChecker DC target… DC is the matrix of the patch digital counts following a spatial correction…)
wherein the test measurement data matrix is an Nx(M*P) matrix (Page 2 DC is the matrix of the patch digital counts following a spatial correction. The subscript m represents the number of channels, in this case, 31 channels. The number of pixels per patch and the number of patches are represented by p and n, respectively). Day discloses m, p, and n, which correspond to N, P and M, respectively.
comprising rows corresponding to N channels in the MIS (Page 2, m represents the number of channels) and columns corresponding to the P pixels (Page 2 The number of pixels per patch …represented by p) in each of the M color samples of the at least one color chart (Page 2, the number of patches are represented by … n)
Therefore, from the teaching of Day, it would have been obvious at the time of filing to specify the abovementioned limitation in order allow for the use of more available data when generating the transformation.
Claim(s) 6 is/are rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20130033585A1; February 7, 2013) in view of Chatterjee (US 2018/0122061A1; May 3, 2018) in view of Bhatti (US 2012/0099788 A1; April 26, 2012).
Regarding claim 6, Li in view of Chatterjee teach the method of claim 2, but fail to teach does not teach wherein the obtaining of the test measurement data comprises obtaining average data of measurement data of P pixels with respect to each of M color samples in the at least one color chart, and wherein the test measurement data matrix is an NxM matrix comprising rows corresponding to N channels in the MIS and columns corresponding to the M color samples of the at least one color chart.
Chatterjee teaches a multispectral image sensor (MIS) (Paragraph 37 -the cameras can be multispectral Paragraph 33)
Therefore, from the teaching of Chatterjee, it would have been obvious at the time of filing to specify the abovementioned limitation in order allow for color calibration of a multispectral image that would yield an MIS with increased detection accuracy.
Li in view Chatterjee do not teach wherein the obtaining of the test measurement data comprises obtaining average data of measurement data of P pixels with respect to each of M color samples in the at least one color chart, and wherein the test measurement data matrix is an NxM matrix comprising rows corresponding to N channels in the MIS and columns corresponding to the M color samples of the at least one color chart.
Bhatti teaches wherein the obtaining of the test measurement data comprises obtaining average data of measurement data of P pixels with respect to each of M color samples in the at least one color chart, and wherein the test measurement data matrix is an NxM matrix comprising rows corresponding to N channels in the camera and columns corresponding to the M color samples of the at least one color chart.
(Column 7 Lines 56-63 - The chart patches are extracted (operation 620), the color pixel values of each patch are averaged (operation 630), and their mean values are compared to reference chart values (sRGB triplets) (operation 640). A 3.times.4 color transform )
Therefore from the teaching of Bhatti, it would have been obvious at the time of filing to specify the abovementioned limitation since it is a known method of obtaining test measurement data in a color calibration method that would allow for increased imaging accuracy once calibrated.
Claim(s) 9 is rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20130033585A1; February 7, 2013) in view of Chatterjee (US 2018/0122061A1; May 3, 2018) in view of Kuzio (Toward Practical Spectral Imaging beyond a Laboratory Contex; December 13, 2022).
Regarding claim 9, Li in view of Chatterjee teach the method of claim 2. Li further teaches wherein the at least one lighting environment comprises Q lighting environments, (Paragraph 23, 25)
wherein the obtaining of the test measurement data comprises measuring the at least one color chart using the test sensor under each of the Q lighting environments, (Paragraph 26, 28)
wherein the obtaining of the reference measurement data comprises measuring the at least one color chart using the reference sensor under each of the Q lighting environments, and (Paragraph 23, 25-28)
Chatterjee teaches a multispectral image sensor (MIS) (Paragraph 37, The camera(s) can be multi-sensor (e.g. 3ccd, RGB+Monochrome) or multi-spectral. Paragraph 33).
Therefore, at the time of filing, one of ordinary skill in the art would have recognized that applying the abovementioned teaching to the system of Li would have yielded predictable results and doing so would have been recognized as resulting in an improved system that would allow for calibration of multispectral image sensors.
Liu in view of Chatterjee fail to teach the at least one transformation model comprises an (N*Q)x(N*Q) matrix comprising rows corresponding to the Q lighting environments and N channels of the reference MIS, and columns corresponding to the Q lighting environments and N channels of the test MIS
Kuzio teches the at least one transformation model comprises an (N*Q)x(N*Q) matrix comprising rows corresponding to the Q lighting environments and N channels of the reference MIS, and columns corresponding to the Q lighting environments and N channels of the test MIS (Page 6, Page 4, two captures, and can be combined to create a six-channel spectral image stack).
Therefore, at the time of filing, one of ordinary skill in the art would have recognized that applying the abovementioned teaching to the system of Li would have yielded predictable results and doing so would have been recognized as resulting in an improved system that would allow for the combination of measurements acquired under different lighting conditions.
Claim(s) 10 is rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20130033585A1; February 7, 2013) in view of Chatterjee (US 2018/0122061A1; May 3, 2018) in view of Shen (US 2014/0309959 A1; October 16, 2014).
Regarding claim 10, Li in view of Chatterjee teach the method of claim 2. Li further teaches the at least one transformation model is generated… based on the test measurement data and the reference measurement data (Paragraph 26-28).
Shen teaches wherein the at least one transformation model is generated using a neural network based on the test measurement data and the reference measurement data (Paragraph 73-75, a neural network transformation algorithm. Paragraphs 88-90).
Therefore, at the time of filing, one of ordinary skill in the art would have recognized that applying the abovementioned teaching to the system of Li would have yielded predictable results and doing so would have been recognized as resulting in an improved system that would allow for the use of a neural network to generate the transformation model.
Claim(s) 11 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20130033585A1; February 7, 2013) in view of Chatterjee (US 2018/0122061A1; May 3, 2018) in view of Wang (US 2011/0058072 A1; March 10, 2011).
Regarding claim 11, Li in view of Chatterjee teach the method of claim 1, but fail to teach teaches wherein the obtaining of the test measurement data comprises: obtaining first test measurement data by measuring a first color chart provided at a first position in an image frame of the test MIS; and
obtaining second test measurement data by measuring a second color chart provided at a second position in the image frame of the test MIS,
Wang teaches wherein the obtaining of the test measurement data comprises: obtaining first test measurement data by measuring a first color chart provided at a first position in an image frame of the test MIS; and
obtaining second test measurement data by measuring a second color chart provided at a second position in the image frame of the test MIS, (Paragraph 29, 32, 33).
wherein the obtaining of the reference measurement data comprises obtaining first reference measurement data by measuring the first color chart provided at the first position in an image frame of the reference MIS, and (Paragraph 41, 47).
wherein the generating of the at least one transformation model comprises: generating, based on the first test measurement data and the first reference measurement data, a first transformation model configured to transform between measurements corresponding to the first position of the test MIS and measurements corresponding to the first position of the reference MIS; and (Paragraph 38, 41).
generating, based on the second test measurement data and the first reference measurement data, a second transformation model configured to transform between measurements corresponding to the second position of the test MIS and measurements corresponding to the first position of the reference MIS (Paragraph 38, 41, 47).
Therefore, at the time of filing, one of ordinary skill in the art would have recognized that applying the abovementioned teaching to the system of Li would have yielded predictable results and doing so would have been recognized as resulting in an improved system that would allow for the calculation of separate transformations for measurements obtained at different positions.
Regarding claim 12, Wang teaches wherein the generating of the at least one transformation model further comprises generating a third transformation model corresponding to a third position that is different from the first position and the second position by interpolating the first transformation model and the second transformation model (Paragraph 43, 41, 47).
Therefore, at the time of filing, one of ordinary skill in the art would have recognized that applying the abovementioned teaching to the system of Li would have yielded predictable results and doing so would have been recognized as resulting in an improved system that would allow for the modeling of a third position by interpolating.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
-US-20120044540-A1 teaches color management and calibration.
-US 20220156899 A1 teaches a method for processing image data.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to GISSELLE GUTIERREZ whose telephone number is (571)272-4672. The examiner can normally be reached M-F 8-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Uzma Alam can be reached at 571-272-3995. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GISSELLE GUTIERREZ/
Examiner
Art Unit 2884
/UZMA ALAM/Supervisory Patent Examiner, Art Unit 2884
Prosecution Insights