Detailed Action
Notice of Pre-AIA or AIA Status
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
2. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
3. Claims 1-2, 4-9, and 11-19 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Soffair et al. US 2024/0095871 A1.
4. Regarding claim 1, claim 1 is rejected for reasons related to claim 8 (see claim 8 below).
5. Regarding claim 2, claim 2 is rejected for reasons related to claim 9 (see claim 9 below).
6. Regarding claim 4, claim 4 is rejected for reasons related to claim 11 (see claim 11 below).
7. Regarding claim 5, claim 5 is rejected for reasons related to claim 12 (see claim 12 below).
8. Regarding claim 6, claim 6 is rejected for reasons related to claim 13 (see claim 13 below).
9. Regarding claim 7, claim 7 is rejected for reasons related to claim 14 (see claim 14 below).
10. Regarding claim 8, an apparatus (…Soffair, in [0033], teaches an electronic device 10; Fig. 1…), comprising:
a memory, storing processor-readable code (…[0035] teaches local memory 20 and main memory storage device 22 for storing processor executable instructions; Fig. 1…); and
at least one processor couple to the memory (…[0035] teaches processor core complex 18 operably coupled with local memory 20 and main memory storage device 22; wherein an image processing circuitry 28 may be part of the processor core complex 18 as taught in [0034]; Fig. 1…), the at least one processor being configured to:
receive image data (…wherein [0050] teaches that image processing circuit 28 receives source image data 48 from an image data source 38…),
one or more input image frames (…[0051] teaches the image data source 38 may include captured images as source image data…);
receive a regions map indicating one or more regions in the image data (…wherein [0032] teaches groupings of pixel values of input image data formatted into tiles; [0050] teaches that pixel data may be referenced to a grouping of sub-pixels Fig. 10…), wherein
the regions map associates each region with one of one or one or more image parameters (…wherein [0030] teaches the image processing circuitry utilizes configuration data to generate a mapping from input image data; further [0060] teaches tiles 112 as groupings of pixel values of input image data (wherein configuration data may include information as such parameters, as taught in [0056])…);
assign, based on the regions map, each coordinate of a subset of coordinates for an output image frame to one of the one or more regions; determine, for each coordinate of the subset of coordinates for the output image frame, one or more input pixels from a corresponding location in one of the one or more input image frames having an image parameter associated with a region assigned to the coordinate (…wherein [0057] teaches that based on configuration data, mapping data 84 is generated correlating output pixel values to pixel values of input image data; further, [0060] teaches input data being fetched as tiled sections (112) wherein the tiles 112 are groupings of pixel values; further, input image data is fetched with regards to a virtual curve which indicates an output row (wherein the each tile is viewed as a subset of coordinates) mapped to a source image and based on mapping data…); and
determine the output image frame by warping the one or more input pixels to align towards the subset of coordinates for the output image frame (…wherein [0054-0055] teach the warping of input image data so to form an output image…).
11. Regarding claim 9, Soffair teaches the apparatus of claim 8 (see claim 8 above), wherein the at least one processor is configured to
determine the output image frame by warping the one or more input pixels by:
interpolating the one or more input pixels (…wherein [0065] teaches that mapped input pixels and support pixels are interpolated so to generate warped image data; process block 190; Fig. 14…).
12. Regarding claim 11, Soffair teaches the apparatus of claim 8 (see claim 8 above), wherein the at least one processor is further configured to
determine the output image frame by performing one or more of:
a lens distortion correction on the one or more input pixels (…wherein [0055] teaches a warp block 52 for warping one or more sets of input image data so to account for input distortions (e.g. camera lens distortion)…).
13. Regarding claim 12, Soffair teaches the apparatus of claim 8 (see claim 8 above), wherein the one or more input image frames comprise
at least a first input image frame having a first image resolution and a second input image frame having a second image resolution, wherein the one or more regions comprises
at least a first region associated with the first image resolution and a second region associated with the second image resolution, wherein the at least one processor is configured to
determine the output image frame by warping the one or more input pixels by blending at least one of the one or more input pixels from the first input image frame with at least one of the one or more input pixels from the second input image frame (…Soffair, in [0055], teaches that warp block 52 of the image processing circuitry 28 may warp one or more sets of input image data to account for input/output distortions or to achieve a common image space for blending; wherein warping sub-blocks are used to warp input image data so to generate warped image data. [0058] further teaches that different components of an input image data may have different resolutions. Thus, Soffair is viewed as teaching different image resolutions corresponding to different regions, also…).
14. Regarding claim 13, Soffair teaches the apparatus of claim 8 (see claim 8 above), wherein the one or more image parameters comprises
at least two different image resolutions (…[0058] teaches that different components of an input image data may have different resolutions…), wherein
the one or more regions includes a region of interest, wherein image data associated with the region of interest is of a higher resolution than other image data (…wherein [0053] teaches that for a viewer’s POV relative to a display or an image capturing device, warped image data accounts for curved edges and/or lensing effects and for a particular type of display different portion of the display are displayed at different resolution (pending on viewer’s gaze, which is viewed as a region of interest); wherein warp block 52 takes into account the resolution at different portions of the display when determining the mapping between input image data and warped image data…).
15. Regarding claim 14, Soffair teaches the apparatus of claim 13 (see claim 13 above), wherein the at least one processor is further configured to:
determine, via an eye tracking sensor, the region of interest in an overlapping field of view (FOV) shared by the one or more input image frames (…wherein [0027] teaches a viewer’s gaze being determined based on viewer’s determined location relative to eye-tracking. [0028] teaches a displayed image may be generated based on multiple sets of image data which is warped to a common image space prior to blending. As such, a common image space is viewed as an overlapping field of view that is shared between multiple sets of image data…); and
generate the region map based on the region of interest (…wherein [0053] teaches that for a viewer’s POV relative to a display or an image capturing device, warped image data accounts for curved edges and/or lensing effects and for a particular type of display different portion of the display are displayed at different resolution (pending on viewer’s gaze, which is viewed as a region of interest); wherein warp block 52 takes into account the resolution at different portions of the display when determining the mapping between input image data and warped image data…).
16. Regarding claim 15, an image capture device (…Soffair, in [0033], teaches an electronic device 10; Fig. 1. [0044] further teaches the electronic device may include one or more cameras to capture pictures/video; Fig. 2…), comprising:
an image sensor configured to generate image data comprising at least two input image frames (…wherein image data source which includes captured images from the one or more cameras; [0041] further teaches image data being generated by an image sensor (e.g. camera)…);
a memory storing processor-readable code (…[0035] teaches local memory 20 and main memory storage device 22 for storing processor executable instructions; Fig. 1…); and
at least one processor coupled to the memory and to the image sensor (…[0035] teaches processor core complex 18 operably coupled with local memory 20 and main memory storage device 22; wherein an image processing circuitry 28 may be part of the processor core complex 18 as taught in [0034]; Fig. 1…),
the at least one processor configured to:
receive the image data comprising the at least two input image frames (…wherein [0050] teaches that image processing circuit 28 receives source image data 48 from an image data source 38…);
receive a regions map indicating at least two regions in the image data (…wherein [0032] teaches groupings of pixel values of input image data formatted into tiles; [0050] teaches that pixel data may be referenced to a grouping of sub-pixels Fig. 10…), wherein
the regions map associates each region with one of at least two different image parameters (…wherein [0030] teaches the image processing circuitry utilizes configuration data to generate a mapping from input image data; further [0060] teaches tiles 112 as groupings of pixel values of input image data (wherein configuration data may include information as such parameters, as taught in [0056])…);
determine, based on the image data, a set of coordinates for an output image frame (…wherein [0060] teaches the input image data 60 being fetched according to a virtual curve 114 indicative of an output row of the warped image data 62 mapped to the source image space; the output row is viewed as corresponding to a set of coordinates…);
assign, based on the regions map, each coordinate of a set of coordinates for an output image frame to one of the at least two regions; determine, for each coordinate for the output image frame, one or more input pixels from a corresponding location in one of the at least two input image frames having an image parameter associated with a region assigned to the coordinate (…wherein [0057] teaches that based on configuration data, mapping data 84 is generated correlating output pixel values to pixel values of input image data; further, [0060] teaches input data being fetched as tiled sections (112) wherein the tiles 112 are groupings of pixel values; further, input image data is fetched with regards to a virtual curve which indicates an output row (wherein the each tile is viewed as a subset of coordinates) mapped to a source image and based on mapping data…); and
determine the output image frame by warping the one or more input pixels to align towards the set of coordinates for the output image frame and interpolating the one or more input pixels after warping the one or more input pixels (…wherein [0054-0055] teach the warping of input image data so to form an output image; [0065] teaches that mapped input pixels and support pixels are interpolated so to generate warped image data; process block 190; Fig. 14…)…).
17. Regarding claim 16, claim 16 is rejected for reasons related to claim 11 (see claim 11 above).
18. Regarding claim 17, claim 17 is rejected for reasons related to claim 12 (see claim 12 above).
19. Regarding claim 18, claim 18 is rejected for reasons related to claim 13 (see claim 13 above).
20. Regarding claim 19, claim 19 is rejected for reasons related to claim 14 (see claim 14 above).
Claim Rejections - 35 USC § 103
21. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
22. Claims 3 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Soffair et al. US 2024/0095871 A1 in view of Chong et al. US 2020/0082496 A1.
23. Regarding claim 3, claim 3 is rejected for reasons related to claim 10 (see claim 10 below).
24. Regarding claim 10, Soffair teaches the apparatus of claim 8 (see claim 8 above), wherein the at least one processor is configured to assign each coordinate by:
determining, a set of coordinates for the output image frame, wherein the set of coordinates includes the subset of coordinates (…wherein [0060] teaches the input image data 60 being fetched according to a virtual curve 114 indicative of an output row of the warped image data 62 mapped to the source image space; the output row is viewed as corresponding to a set of coordinates…); and
determining, via an inverse transformation of the set of coordinates, a virtual set of coordinates that is scaled to the regions map (…[0060] teaches input image data is fetched with regards to a virtual curve which indicates an output row, wherein the virtual curve is indicative of a first row (see Fig. 10).
An inverse transformation is not taught as part of a mapping process.
However, Chong teaches a method and system for generating a warp map for projecting on a non-planar surface (Chong in paragraph [0015] teaches a calibration pattern being projected on the non-planar surface by applying an inverse transform to each region of the non-planar surface and each transform mapping pixels of the projection to pixels of the camera in accordance with an initial warp map…).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Soffair with an inverse transform process employed in the mapping as seen in Chong to output exact pixel locations on non-planar surface for projection to generate an improved warp map.
25. Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Soffair et al. (US 2024/0095871 A1) in view of Mahbub (WO2023044233A1).
26. Regarding claim 20, Soffair teaches the image capture device of claim 18 (see claim 18 above), further comprising
a facial recognition sensor, wherein
the at least one processor is further configured to:
determine, via the facial recognition sensor, a region of interest; and
generate the regions map based on the region of interest (…wherein [0030] teaches
the image processing circuitry utilizes configuration data to generate a mapping
from input image data; [0032] teaches groupings of pixel values of input image
data formatted into tiles; [0050] teaches that pixel data may be referenced to a
grouping of sub-pixels Fig. 10; further [0060] teaches tiles 112 as groupings of
pixel values of input image data (wherein configuration data may include
information as such parameters, as taught in [0056]); [0053] teaches that
for a viewer’s POV relative to a display or an image capturing device, warped
image data accounts for curved edges and/or lensing effects and for a particular
type of display different portion of the display are displayed at different resolution
(pending on viewer’s gaze, which is viewed as a region of interest), wherein
warp block 52 takes into account the resolution at different portions of the display
when determining the mapping between input image data and warped image data
…).
Soffair does not further teach
a facial recognition sensor,
wherein the at least one processor is further configured to:
determine, via the facial recognition sensor, a region of interest.
However, Mahbub teaches
a facial recognition sensor (…wherein Mahbub, in [0152], teaches a detection
system 500, e.g. a face detection/recognition in which an object detected is analyzed for detection…), wherein
the at least one processor is further configured to:
determine, via the facial recognition sensor, a region of interest (…[0151]
teaches detection system 500 determines a ROI corresponding to an object being detected.
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention (AIA ) to modify the invention of Soffair with that the generation of a mapping from input data image data, as taught by Mahbub, can be implemented based on the recognition of a face, similar to the determination of a region of interest which depends on a viewer’s gaze, thereby providing a better resolution on a detected face of an image.
Conclusion
27. Any inquiry concerning this communication or earlier communications from the examiner should be directed to SURAFEL YILMAKASSAYE whose telephone number is (703)756-1910. The examiner can normally be reached Monday-Friday 8:30am-5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, TWYLER HASKINS can be reached at (571)272-7406. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SURAFEL YILMAKASSAYE/Examiner, Art Unit 2639
/TWYLER L HASKINS/Supervisory Patent Examiner, Art Unit 2639
Prosecution Insights