DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 11/20/2025 have been fully considered but they are not persuasive.
Regarding claim 1, and similarity for claim 6, applicant argues that “Aoki fails to disclose or suggest reading out a given line in the first set of lines that comprises colour filters of each of the at least three different colours that is a row, and reading out another given line in the first set of lines that comprises colour filters of each of the at least three different colours that is a column.”. The examiner notes that the claim does not recite “…reading out another given line in the first set of lines…”.
Regarding claim 10, and similarly for claim 13, applicant argues that “Aoki fails to disclose or suggest a colour filter array comprising sets of repeating rows or columns, where each set comprises N rows or columns that repeat consecutively, and where the processor is configured to read out at most N-1 rows or columns from each set.” while referring to Figure 21. The examiner notes that Figure 21 was not relied upon in the rejection, but instead Figures 22 and 23 were cited. The examiner further notes that Aoki was shown to perform “imaging is performed by ½ thinning reading in which imaging is performed by skipping one line at a time, par. 118 and 152, where one set of lines would necessarily be even and the other set would necessarily be odd in order to be ½ thinning” and “when imaging is performed by ½ thinning reading in which imaging is performed by skipping one line at a time” as seen in cited par. 118 and 152, which relate to cited par. 22 and 23 that was used to show the repeating rows and columns in the rejection of claim 10.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1, 3, 6, 7, 10, and 13 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Aoki et al. (U.S. Pub. No. 20230254600).
Regarding claim 1, Aoki discloses:
An imaging system comprising:
an image sensor comprising: a plurality of photo-sensitive cells arranged on a photo-sensitive surface of the image sensor (the sensor unit 10 has a pixel array in which pixels including at least one photoelectric conversion element are arranged in a matrix, and the light receiving surface is formed by each pixel arranged in a matrix in the pixel array, par. 73); and
a colour filter array comprising colour filters of at least three different colours (a color filter is provided for each pixel included in the sensor unit 10 and the pixel data has color information of red (R), green (G), and blue (B) having a repeating pattern in the row and column direction, the visual recognition processing unit 14 can execute demosaic processing, white balance processing, and the like, par. 80 and Figs. 22 and 23); and
at least one processor (sensor control unit 11 includes, for example, a microprocessor, controls the reading out of pixel data from the sensor unit 10, and outputs pixel data based on each pixel signal read out from each pixel included in the frame, par. 78) configured to:
read out image data from the image sensor (vertical scanning unit 102 transmits the control signal such as a drive pulse at the time of reading out the pixel signal from the pixel, to the pixel array unit 101 via the pixel signal line 106 under the control of the control unit 1100, and the pixel signal read out from the pixel is transmitted to the AD conversion unit 103 via the vertical signal line VSL, par. 97), wherein when reading out, the at least one processor is configured to employ subsampling in at least a region of the photo-sensitive surface, by:
reading out the image data from those photo-sensitive cells that correspond to a first set of lines in the colour filter array, wherein a given line in the first set comprises colour filters of each of the at least three different colours (G pixels are arranged in every other column in the direction orthogonal to the readout direction of the line data and columns in which R and B pixels are alternately arranged are arranged between the columns in which the G pixels are arranged, and as illustrated in FIG. 22, in the columns in which the R and B pixels are alternately arranged, the arrangement order of the R and B pixels is different for each column, and even in a case where data is read out in any line such as the line data L200 or L202, all types of color information can be acquired, par. 184 and Fig. 22, or R, B, and G pixels are alternately arranged in the columns in the direction orthogonal to the readout direction of the line data and the arrangement order of R, B, and G pixels is different for each column, and even in a case where data is read out in any line such as the line data L230 or L232, all types of color information can be acquired, par. 185 and Fig. 23); and
skipping read out from those photo-sensitive cells that correspond to a second set of lines in the colour filter array, wherein the first set of lines comprises one of: odd lines in the colour filter array, even lines in the colour filter array, while the second set of lines comprises another of: the odd lines, the even lines (imaging is performed by ½ thinning reading in which imaging is performed by skipping one line at a time, par. 118 and 152, where one set of lines would necessarily be even and the other set would necessarily be odd in order to be ½ thinning); and
process the image data to generate an image (visual recognition processing unit 14 can execute demosaic processing, and the image data subjected to the image processing of the pixel data by the visual recognition processing unit 14 is supplied to the output control unit 15, par. 80);
wherein the given line in the first set is a row of the colour filter array (G pixels are arranged in every other column in the direction orthogonal to the readout direction of the line data and columns in which R and B pixels are alternately arranged are arranged between the columns in which the G pixels are arranged, and as illustrated in FIG. 22, in the columns in which the R and B pixels are alternately arranged, the arrangement order of the R and B pixels is different for each column, and even in a case where data is read out in any line such as the line data L200 or L202, all types of color information can be acquired, par. 184 and Fig. 22, or R, B, and G pixels are alternately arranged in the columns in the direction orthogonal to the readout direction of the line data and the arrangement order of R, B, and G pixels is different for each column, and even in a case where data is read out in any line such as the line data L230 or L232, all types of color information can be acquired, par. 185 and Fig. 23), and
wherein another given line in the first set comprising colour filters of each of the at least three different colours is a column of the colour filter array (each column of pixels in Fig. 23 has R, G, and B pixels, par. 185 and Fig. 23).
Regarding claim 3, Aoki further discloses:
the at least one processor is configured to select the first set of lines and the second set of lines as sets of at least one of: rows, columns, based on whether colour filters of each of the at least three different colours are arranged in rows, columns, or both rows and columns (imaging is performed by ½ thinning reading in which imaging is performed by skipping one line at a time, where in a case where data is read out in any line such as the line data L200 or L202, all types of color information can be acquired or in a case where data is read out in any line such as the line data L230 or L232, all types of color information can be acquired, par. 152, 184, 185).
Regarding claim 6, see the rejection of claim 1 and note that the limitations of claim 6 were shown.
Regarding claim 7, see the rejection of claims 6 and 3 and note that the limitations of claim 7 were shown.
Regarding claim 10, Aoki discloses:
An imaging system comprising:
an image sensor comprising:
a plurality of photo-sensitive cells arranged on a photo-sensitive surface of the image sensor (the sensor unit 10 has a pixel array in which pixels including at least one photoelectric conversion element are arranged in a matrix, and the light receiving surface is formed by each pixel arranged in a matrix in the pixel array, par. 73); and
a colour filter array comprising sets of repeating rows or columns, wherein a given set of repeating rows or columns comprises N rows or columns that repeat consecutively (a color filter is provided for each pixel included in the sensor unit 10 and the pixel data has color information of red (R), green (G), and blue (B) having a repeating pattern in the row and column direction, the visual recognition processing unit 14 can execute demosaic processing, white balance processing, and the like, par. 80 and Figs. 22 and 23); and
at least one processor (sensor control unit 11 includes, for example, a microprocessor, controls the reading out of pixel data from the sensor unit 10, and outputs pixel data based on each pixel signal read out from each pixel included in the frame, par. 78) configured to:
read out image data from the image sensor (G pixels are arranged in every other column in the direction orthogonal to the readout direction of the line data and columns in which R and B pixels are alternately arranged are arranged between the columns in which the G pixels are arranged, and as illustrated in FIG. 22, in the columns in which the R and B pixels are alternately arranged, the arrangement order of the R and B pixels is different for each column, and even in a case where data is read out in any line such as the line data L200 or L202, all types of color information can be acquired, par. 184 and Fig. 22, or R, B, and G pixels are alternately arranged in the columns in the direction orthogonal to the readout direction of the line data and the arrangement order of R, B, and G pixels is different for each column, and even in a case where data is read out in any line such as the line data L230 or L232, all types of color information can be acquired, par. 185 and Fig. 23), wherein when reading out, the at least one processor is configured to employ subsampling in at least a region of the photo-sensitive surface, by:
reading out the image data from those photo-sensitive cells that correspond to at most N-1 rows or columns out of the N rows or columns in each set of repeating rows or columns (imaging is performed by ½ thinning reading in which imaging is performed by skipping one line at a time, par. 118 and 152, where one set of lines would necessarily be even and the other set would necessarily be odd in order to be ½ thinning); and
skipping read out from those photo-sensitive cells that correspond to a remainder of the N rows or columns in each set of repeating rows or columns (when imaging is performed by ½ thinning reading in which imaging is performed by skipping one line at a time, par. 118 and 152); and
process the image data to generate an image (visual recognition processing unit 14 can execute demosaic processing, and the image data subjected to the image processing of the pixel data by the visual recognition processing unit 14 is supplied to the output control unit 15, par. 80).
Regarding claim 13, see the rejection of claim 10 and note that the limitations of claim 13 were shown.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 4, 5, 8, 9, 11, 12, 14, and 15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Aoki et al. (U.S. Pub. No. 20230254600) in view of Berkovich (U.S. Pub. No. 20240007771).
Regarding claim 4, Aoki is silent with regards to processor is configured to: obtain information indicative of a gaze direction of a user; determine a gaze region and a peripheral region in the photo-sensitive surface of the image sensor, based on the gaze direction; and select the peripheral region as said region of the photo-sensitive surface in which the subsampling is to be employed. Berkovich discloses processor (processor performing the operations, par. 91-92) is configured to: obtain information indicative of a gaze direction of a user (gaze vector or eye orientation received from eye tracking system 114, par. 26 and 29); determine a gaze region and a peripheral region in the photo-sensitive surface of the image sensor, based on the gaze direction (processing logic 120 may base readout configuration 104 and/or ROI configuration 108 on a gaze vector or eye orientation received from eye tracking system 114, where ROI configuration 108 may include a quantity 146, a sampling ratio 148, a size 150, and a location 152 of the ROIs in pixel array 128, where a first ROI, second ROI, and third ROI are defined and where a fourth ROI may be designated as ROI level 1, may be the smallest, may encompass a foveated region (e.g., a region of focus of an eye), and may have a sampling ratio of 1:1 where each pixel is included in the image data for the fourth ROI, and fourth ROI level 262 is centered around a fovea center 264, which is an indication of the point of focus of a gaze angle or eye orientation for a user, par. 29, 36, 45-47, and Figs. 2B-2C); and select the peripheral region as said region of the photo-sensitive surface in which the subsampling is to be employed (ROI level 1, may be the largest, and may have a sampling ratio of 8:1 where one out of eight pixels are included in the image data for the first ROI, a second ROI may be designated as ROI level 2, may be the second largest, and may have a sampling ratio of 4:1 where one out of four pixels are included in the image data for the second ROI, a third ROI may be designated as ROI level 3, may be the third largest, and may have a sampling ratio of 2:1 where one out of two pixels are included in the image data for the third ROI, where fourth ROI is designated as ROI level 1, par. 36). As can be seen in par. 37 this is advantageous in that the ROI configurations define how ROI data is grouped and compressed into data frames for transmission. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include processor is configured to: obtain information indicative of a gaze direction of a user; determine a gaze region and a peripheral region in the photo-sensitive surface of the image sensor, based on the gaze direction; and select the peripheral region as said region of the photo-sensitive surface in which the subsampling is to be employed.
Regarding claim 5, Aoki is silent with regards to processor is configured to: obtain information indicative of a gaze direction of a user; and determine a gaze region and a peripheral region in the photo-sensitive surface of the image sensor, based on the gaze direction, wherein when employing the subsampling, the at least one processor is configured to employ the subsampling in an entirety of the photo-sensitive surface with a first subsampling density in the gaze region and with a second subsampling density in the peripheral region, the first subsampling density being higher than the second subsampling density. Berkovich discloses processor (processor performing the operations, par. 91-92) is configured to: obtain information indicative of a gaze direction of a user (gaze vector or eye orientation received from eye tracking system 114, par. 26 and 29); and determine a gaze region and a peripheral region in the photo-sensitive surface of the image sensor, based on the gaze direction (processing logic 120 may base readout configuration 104 and/or ROI configuration 108 on a gaze vector or eye orientation received from eye tracking system 114, where ROI configuration 108 may include a quantity 146, a sampling ratio 148, a size 150, and a location 152 of the ROIs in pixel array 128, where a first ROI, second ROI, and third ROI are defined and where a fourth ROI may be designated as ROI level 1, may be the smallest, may encompass a foveated region (e.g., a region of focus of an eye), and may have a sampling ratio of 1:1 where each pixel is included in the image data for the fourth ROI, and fourth ROI level 262 is centered around a fovea center 264, which is an indication of the point of focus of a gaze angle or eye orientation for a user, par. 29, 36, 45-47, and Figs. 2B-2C), wherein when employing the subsampling, the at least one processor is configured to employ the subsampling in an entirety of the photo-sensitive surface with a first subsampling density in the gaze region and with a second subsampling density in the peripheral region, the first subsampling density being higher than the second subsampling density (ROI level 1, may be the largest, and may have a sampling ratio of 8:1 where one out of eight pixels are included in the image data for the first ROI, a second ROI may be designated as ROI level 2, may be the second largest, and may have a sampling ratio of 4:1 where one out of four pixels are included in the image data for the second ROI, a third ROI may be designated as ROI level 3, may be the third largest, and may have a sampling ratio of 2:1 where one out of two pixels are included in the image data for the third ROI, where fourth ROI is designated as ROI level 1, par. 36). As can be seen in par. 37 this is advantageous in that the ROI configurations define how ROI data is grouped and compressed into data frames for transmission. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to include processor is configured to: obtain information indicative of a gaze direction of a user; and determine a gaze region and a peripheral region in the photo-sensitive surface of the image sensor, based on the gaze direction, wherein when employing the subsampling, the at least one processor is configured to employ the subsampling in an entirety of the photo-sensitive surface with a first subsampling density in the gaze region and with a second subsampling density in the peripheral region, the first subsampling density being higher than the second subsampling density.
Regarding claim 8, see the rejection of claims 6 and 4 and note that the limitations of claim 8 were shown.
Regarding claim 9, see the rejection of claims 6 and 5 and note that the limitations of claim 9 were shown.
Regarding claim 11, see the rejection of claims 10 and 4 and note that the limitations of claim 11 were shown.
Regarding claim 12, see the rejection of claims 10 and 5 and note that the limitations of claim 5 where shown.
Regarding claim 14, see the rejection of claims 13 and 11 and note that the limitations of claim 14 were shown.
Regarding claim 15, see the rejection of claims 13 and 12 and note that the limitations of claim 15 were shown.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS G GILES whose telephone number is (571)272-2824. The examiner can normally be reached M-F 6:45AM-3:15PM EST (HOTELING).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Twyler Haskins can be reached at 571-272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS G GILES/Primary Examiner, Art Unit 2639