Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statements (IDS) submitted on 08/25/2025 and 11/03/2025 are in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statements are being considered by the examiner.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 102(a)(1) as being anticipated by us patent application publication no. 2019/0379893 to Krishnan (hereinafter Kris).
For claim 1, Kris as applied discloses an apparatus comprising:
a processor (see, e.g., pars. 21 and 108-115 and FIG. 9, which teach using a processor);
a memory storing instructions that, when executed by the processor, perform a method for compressing a source image (see, e.g., pars. 108-115 and FIG. 9, which teach using a memory), comprising:
receiving the source image, the source image having a source resolution (see, e.g., pars. 24-25 and FIGS. 1 and 3A, which teach that the received input/original image having a w0xh0 resolution/size); and
mapping a source pixel of the source image to a target pixel of a target image using a distortion function (see, e.g., pars. 24-33 and FIGS. 1, 3A and 4A, which teach sampling pixels of the original image to be the pixels of a downsampled image using a downsampling function), the target image having a lower resolution than the source resolution (see, e.g., pars. 24-33 and FIGS. 1 and 4A, which teaches performing multi-segment downsampling on the input image to generate a lower resolution downsampled image),
wherein the distortion function defines a mapping (see, e.g., pars. 24-33 and FIGS. 1, 3A and 4A, which teach sampling pixels of the original image to be the pixels of a downsampled image using a downsampling function), the mapping comprising a one-to-one source-to-target pixel mapping within a foveal region (see, e.g., pars. 27 and 31-33 and FIG. 4A, which teach keeping the down sampling ratio in the ROI as 1 so that the sampling points lie exactly on the pixel locations of the original image and no resampling is needed), and the mapping comprising a more-than-one-to-one source-to-target pixel mapping outside of the foveal region (see, e.g., pars. 25-26 and 31-33 and FIG. 4A, which teach that the image is sampled more sparsely sampled outside the ROI), and wherein the foveal region is a defined area of pixels (see, e.g., pars. 25-27 and 31 and FIG. 3A, which tech defining a rectangular area with threshold height and width as the ROI).
For claim 2, Kris as applied discloses that the source image comprises at least one pixel with at least one associated pixel value (see, e.g., pars. 32-33 and FIGS. 4A-B, which teach sampling pixel locations of the original image) and wherein the method further comprises:
in response to the target pixel being within the foveal region, defining an associated pixel value of the target pixel to be equivalent to an associated pixel value of the source pixel (see, e.g., pars. 27, 33, 36, and 38-40 and FIGS. 4A-B and 5A-B, which teach that the sampling points in the ROI lie exactly on the pixel locations of the original image and no resampling is needed); and
in response to the target pixel being outside of the foveal region, defining an associated pixel value of the target pixel using a downsampling technique (see, e.g., pars. 32-33 and 38-40 and FIGS. 4A and 5A, which teach that sampling locations outside the ROI are more sparsely spread than the original pixel locations).
For claim 3, Kris as applied discloses that the downsampling technique comprising a filtering technique (see, e.g., pars. 30, 32 and 36), the filtering technique comprising any of:
linear filtering comprising defining the associated pixel value of the target pixel to be an average of pixel values associated with source pixels mapped to the target pixel (see, e.g., pars. 30, 32 and 36, which teach using the linear spacing and bilinear interpolation),
defining the associated pixel value of the target pixel to be a pixel value associated with a single source pixel mapped to the target pixel,
defining the associated pixel value of the target pixel to be a sum of source pixels mapped to the target pixel,
nearest neighbor filtering,
anisotropic filtering,
Lanczos filtering.
For claim 4, Kris as applied discloses comprising encoding the target image to produce an encoded target image (see, e.g., pars. 24, 33, which teach encoding the downsampled image to generate a bitstream) and sending the encoded target image to a head-mounted device for display (see, e.g., pars. 34, 104-106 and 109, which teach sending the encoded downsampled image to a display, e.g., a display of a head mounted display device).
For claim 5, Kris as applied discloses that the foveal region is defined using at least one characteristic of a device by which the target image is to be decompressed and displayed (see, e.g., par. 31, which teaches that the ROI parameters are determined based on the threshold that is based on the screen size).
For claim 6, Kris as applied discloses that the target image is for decompression and display on a display of a head-mounted device (see, e.g., pars. 34-35, 79, 104-106, and 109, which teach sending the encoded downsampled image to be decoded/decompressed and displayed on a display, e.g., a display of a head mounted display device), and
wherein the foveal region is defined using at least one of (the examiner interprets the following condition disjunctively):
a position of the display of the head-mounted device (see, e.g., pars. 99 and 104-106 and FIGS. 8A and B, which teach that the ROI is defined using the gaze direction with respect to the position of the display screen on the user’s head) and
a lens type of at least one lens of the head-mounted device.
For claim 7, Kris as applied discloses that the foveal region is defined using at least one of (the examiner interprets the following condition disjunctively):
a gaze direction of a user of a device by which the target image is to be decompressed and displayed (see, e.g., pars. 37 and 94-106 and FIGS. 8A-B, which teach that the ROI is determined based on the gaze tracking system),
at least one characteristic of a network via which the target image is to be transmitted,
an attention of a user of a device by which the target image is to be decompressed and displayed,
a defined importance factor of an element of the source image.
For claim 8, Kris as applied discloses that the foveal region is one of:
predefined prior to the receiving of the source image (see, e.g., par. 26, which teaches determining the ROI parameters based on the predefined factors such as the required bit rate of the compressed image and the degree of quality loss outside the ROI) and
dynamically defined by the method (see, e.g., pars. 24-31, which teach determining the ROI parameters based on the received frames).
For claim 9, Kris as applied discloses that the distortion function is separable along a vertical and a horizontal dimension of the source and target images (see, e.g., pars. 33 and 39 and FIGS. 4A-B and 5A-B, which teach having different sampling sparsity between the axes), and
wherein the mapping of a source pixel of the source image to a target pixel of a target image using the distortion function comprises determining a target pixel by any of (the examiner interprets the following condition disjunctively):
determining a horizontal position of the target pixel in the target image independently of determining a vertical position of the target pixel in the target image by applying the horizontal part of the separable distortion function to a horizontal position of the source pixel in the source image (see, e.g., pars. 32-33, which teach changing the sample density along each axis so that a density along one axis is sparser than the other),
determining a vertical position of the target pixel in the target image independently of determining a horizontal position of the target pixel in the target image by applying the vertical part of the separable distortion function to a vertical position of the source pixel in the source image (see, e.g., pars. 32-33, which teach changing the sample density along each axis so that a density along one axis is sparser than the other).
For claim 10, Kris as applied discloses that the foveal region is a rectangular region of pixels (see, e.g., pars. 24-25 and FIGS. 3A-B).
For claim 11, Kris as applied discloses that the more than one-to-one source-to-target pixel mapping outside of the foveal region is a linear mapping (see, e.g., pars. 30-32 and 36, and FIGS. 4A-B, which teach using a linear mapping/spacing for sampling pixels) .
For claim 12, Kris as applied discloses that the more than one-to-one source-to-target pixel mapping outside of the foveal region is a quadratic mapping (see, e.g., pars. 38-40 and FIGS. 5A-B, which teach using a quadradic functions for sampling).
For claim 13, Kris as applied discloses that a slope of the quadratic mapping matches a slope of the one-to-one source-to-pixel mapping of the foveal region, at a pixel located at a boundary between the foveal region and the outside of the foveal region (see, e.g., pars. 38-40 and FIGS. 5A-B, which teaches that the sampling locations are more closely tied to the original pixel locations as the samples are closer to the ROI, and in FIGS. 5A-B, the sampling points in the corner of the ROI, i.e., those in the boundary, correspond to the original pixel locations).
For claim 14, Kris as applied discloses a method for compressing a source image for sending to a head-mounted device for decompression and display, the method comprising:
receiving the source image, the source image having a source resolution and comprising at least one pixel with at least one associated pixel value (see, e.g., pars. 24-25 and FIGS. 1 and 3A, which teach that the received input/original image having a w0xh0 resolution/size);
mapping a source pixel of the source image to a target pixel of a target image using a distortion function, the target image having a lower resolution than the source resolution (see, e.g., pars. 24-33 and FIGS. 1, 3A and 4A, which teach sampling pixels of the original image to be the pixels of a downsampled image using a downsampling function),
wherein the distortion function defines a mapping (see, e.g., pars. 24-33 and FIGS. 1, 3A and 4A, which teach sampling pixels of the original image to be the pixels of a downsampled image using a downsampling function), the mapping comprising a one-to-one source-to-target pixel mapping within a foveal region (see, e.g., pars. 27 and 31-33 and FIG. 4A, which teach keeping the down sampling ratio in the ROI as 1 so that the sampling points lie exactly on the pixel locations of the original image and no resampling is needed), and the mapping comprising a more than one-to-one source-to-target pixel mapping outside of the foveal region (see, e.g., pars. 25-26 and 31-33 and FIG. 4A, which teach that the image is sampled more sparsely sampled outside the ROI), and wherein the foveal region is a defined area of pixels (see, e.g., pars. 25-27 and 31 and FIG. 3A, which tech defining a rectangular area with threshold height and width as the ROI);
in response to the target pixel being within the foveal region, defining an associated pixel value of the target pixel to be equivalent to an associated pixel value of the source pixel (see, e.g., pars. 27, 33, 36, and 38-40 and FIGS. 4A and 5A, which teach that the sampling points in the ROI lie exactly on the pixel locations of the original image and no resampling is needed);
in response to the target pixel being outside of the foveal region, defining an associated pixel value of the target pixel using a downsampling technique (see, e.g., pars. 32-33 and 38-40 and FIGS. 4A and 5A, which teach that sampling locations outside the ROI are more sparsely spread than the original pixel locations);
encoding the target image using a hardware encoding unit to produce an encoded target image (see, e.g., pars. 24 and 33, which teach encoding the downsampled image to generate a bitstream); and
sending the encoded target image to the head-mounted device (see, e.g., pars. 34 and 109, which teach sending the encoded data to a display, e.g., a display of a head mounted display device).
For claim 15, Kris as applied discloses that the method at least partially carried out using hardware logic (see, e.g., pars. 80 and 116, which teach using the hardware decoder and network interface).
For claim 16, Kris as applied discloses a method for decompressing a compressed image, comprising:
receiving the compressed image, the compressed image having a target resolution (see, e.g., pars. 34 and 76-77 and FIGS. 2 and 7, which teach receiving the compressed/encoded, downsampled image); and
mapping a target pixel of the compressed image to a source pixel of a source image using a distortion function (see, e.g., pars. 34-35 and FIGS. 3B and 4B, which teach mapping pixels of the decoded image to the pixels of the original image using an upsampling function), the source image having a higher resolution than the target resolution (see, e.g., pars. 34-35, which teach that the original size is reached by upsampling the decoded image),
wherein the distortion function defines a mapping (see, e.g., pars. 26-31 and 35-39 and FIGS. 3B and 4B, which teach using an overall downsampling ratio including ratios for the ROI and regions outside the ROI), the mapping comprising a one-to-one target-to-source pixel mapping within a foveal region (see, e.g., pars. 27, 33 and 36-39 and FIGS. 4B and 5B, which teach keeping the down sampling ratio in the ROI as 1 so that the sampling points lie exactly on the pixel locations of the original image and no resampling is needed), and the mapping comprising one-to-more-than-one target-to-source pixel mapping outside of the foveal region (see, e.g., pars. 25-26 and 35-39 and FIGS. 4B and 5B, which teach that the sampling density is higher for the background compared to the ROI during upsampling), and
wherein the foveal region is a defined area of pixels (see, e.g., pars. 25-27 and 31 and FIGS. 3A-B, 4A-B and 5A-B, which tech defining a rectangular area with threshold height and width as the ROI).
For claim 17, Kris as applied discloses that the compressed image comprises at least one pixel with at least one associated pixel value (see, e.g., pars. 35-40 and FIGS. 4B and 5B), and wherein the method further comprises:
in response to the source pixel being within the foveal region, defining an associated pixel value of the source pixel to be equivalent to an associated pixel value of the target pixel (see, e.g., pars. 35-40 and FIGS. 4B and 5B, which teach upsampling the decoded downsampled image, wherein the associated pixel values lie in the original pixel locations); and
in response to the source pixel being outside of the foveal region, defining an associated pixel value of the source pixel using an upsampling technique (see, e.g., pars. 35-40 and FIGS. 4B and 5B, which teach upsampling the decoded downsampled image, wherein the background is upsampled with a higher density than the ROI).
For claim 18, Kris as applied discloses that the upsampling technique comprising a filtering technique (see, e.g., pars. 30, 32 and 36 and FIG. 4B), the filtering technique comprising any of (the examiner interprets the following condition disjunctively):
linear filtering, defining the associated pixel value of the source pixel to be an average of pixel values associated with pixels within a defined distance of the target pixel in the compressed image (see, e.g., pars. 30, 32 and 36 and FIG. 4B, which teach using the linear spacing and bilinear interpolation),
defining the associated pixel value of the source pixel to be an associated pixel value of the target pixel,
nearest-neighbor filtering,
defining the associated pixel value of the source pixel to be an average of pixel values associated with pixels within a defined distance of the source pixel in the source image wherein a pixel of the source image used for the upsampling technique without an associated value has an associated value defined for the upsampling technique to be an associated pixel value of a pixel of the compressed image that is mapped to the pixel of the source image,
anisotropic filtering,
Lanczos filtering.
For claim 19, Kris as applied discloses that the method at least partially carried out using hardware logic (see, e.g., pars. 80 and 116, which teach using the hardware decoder and network interface).
For claim 20, Kris as applied discloses that the compressed image is encoded (see, e.g., pars. 33-34 and FIGS. 2 and 3B, which teach that the downsampled image is encoded), the method further comprising decoding the compressed image prior to the mapping (see, e.g., pars. 33-34 and FIGS. 2 and 3B, which teach that the encoded image is decoded before upsampling), and the method further comprising displaying the source image on a display of a head-mounted device (see, e.g., pars. 34 and 109 and FIG. 2, which teach that the upsampled, decoded image is displayed on a display, e.g., a display of a head mounted display device).
Additional Citations
The following table lists several references that are relevant to the subject matter claimed and disclosed in this Application. The references are not relied on by the Examiner, but are provided to assist the Applicant in responding to this Office action.
Citation
Relevance
Berkovich et al. (us patent application publication no. 2024/0007771)
Describes systems and readout methods for foveated sensing that may include a pixel array, readout circuitry, and processing logic. In one embodiment, the pixel array may have a number of pixels. The readout circuitry may be configured to read image data from the pixel array for each of the plurality of pixels. The processing logic may be configured to identify a number of regions of interest (ROIs) within the pixel array. The processing logic may be configured to associate the image data for the pixels with one or more ROIs. The processing logic may be configured to arrange the image data into data frames. The data frames may include the image data ordered by ROI. Image data for inactive pixels may be removed from the image data and data frames prior to transmission. The processing logic may be configured to transmit the data frames in an order that is based on the ROIs.
Zhang (us patent application publication no. 2017/0295373)
Describes head mounted display (HMD) devices and more particularly encoding image data at an HMD device. In one embodiment, an HMD device encodes different portions of an image for display with different encoding characteristics based on a user's predicted area of focus as indicated by one or more of a pose of the HMD device and a gaze direction of the user's eye(s) identified at the HMD device. By employing different encoding characteristics, the HMD device supports relatively high-quality encoding while maintaining a relatively small size of the encoded image to allow for transfer of the image to a display panel at a high frame rate. Thus, the HMD device can encode a portion of the image that is expected to be in the user's area of focus at a high resolution, and encode the portion of the image that is expected to be in the user's peripheral vision at a lower resolution.
Bezugly et al. (us patent application publication no. 2022/0318953)
Describes a method for improved compression and filtering of images for foveated transport applications. In one embodiment, the improvements including calculating along at least one axis of the full-resolution image data set, the distance from the foveal point to edges of full-resolution image data set, calculating the distance from each edge of the full resolution data set to the closest point on a foveal region surrounding the foveal point, calculating the distribution of available space in the adjacent peripheral regions of the image using a compression parameter, and calculating a compression curve wherein the compression of both adjacent peripheral regions is such that neither side of the adjacent peripheral regions are compressed more than the other.
Table 1
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. See Table 1 and form 892.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WOO RHIM whose telephone number is (571)272-6560. The examiner can normally be reached Mon - Fri 9:30 am - 6:00 pm et.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at 571-272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WOO C RHIM/Examiner, Art Unit 2676