Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Submission dated 05/26/2026 amends claims 1, 7, 9, 13, 14, and 16. Claims 1-20 are pending.
Response to Arguments
Applicant’s arguments with respect to claim(s) 1-20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over us patent application publication no. 2019/0379893 to Krishnan (hereinafter Kris) in view of us patent application publication no. 2024/0007771 to Berkovich et al. (hereinafter Berkovich).
For claim 1, Kris as applied teaches an apparatus comprising:
a processor (see, e.g., pars. 21 and 108-115 and FIG. 9, which teach using a processor);
a memory storing instructions that, when executed by the processor, perform a method for compressing a source image (see, e.g., pars. 108-115 and FIG. 9, which teach using a memory), comprising:
receiving the source image, the source image having a source resolution (see, e.g., pars. 24-25 and FIGS. 1 and 3A, which teach that the received input/original image having a w0xh0 resolution/size); and
mapping a source pixel of the source image to a target pixel of a target image using a distortion function (see, e.g., pars. 24-33 and FIGS. 1, 3A and 4A, which teach sampling pixels of the original image to be the pixels of a downsampled image using a downsampling function), the target image having a lower resolution than the source resolution (see, e.g., pars. 24-33 and FIGS. 1 and 4A, which teaches performing multi-segment downsampling on the input image to generate a lower resolution downsampled image),
wherein the distortion function defines a mapping (see, e.g., pars. 24-33 and FIGS. 1, 3A and 4A, which teach sampling pixels of the original image to be the pixels of a downsampled image using a downsampling function), the mapping comprising a one-to-one source-to-target pixel mapping within a foveal region (see, e.g., pars. 27 and 31-33 and FIG. 4A, which teach keeping the down sampling ratio in the ROI as 1 so that the sampling points lie exactly on the pixel locations of the original image and no resampling is needed), and the mapping comprising a more-than-one-to-one source-to-target pixel mapping outside of the foveal region (see, e.g., pars. 25-26 and 31-33 and FIG. 4A, which teach that the image is sampled more sparsely sampled outside the ROI), and wherein the foveal region is a defined area of pixels (see, e.g., pars. 25-27 and 31 and FIG. 3A, which tech defining a rectangular area with threshold height and width as the ROI).
While Kris as applied does not explicitly teach, Berkovich in the analogous art teaches that the foveal region is defined using one or more of: a characteristic of a network via which the target image is to be transmitted, an attention of a user of a device by which the target image is to be decompressed and displayed (see, e.g., par. 13 of Berkovich teaches that the foveated sensing may be implemented based on factors such as user state including user attention), or an importance factor of an element of the source image.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kris to define the foveal region using additional factor taught by Berkovich because doing so would improve the use of system resource in an artificial reality system such as an HMD (see, e.g., par. 13 of Berkovich).
For claim 2, Kris in view of Berkovich teaches that the source image comprises at least one pixel with at least one associated pixel value (see, e.g., pars. 32-33 and FIGS. 4A-B of Kris, which teach sampling pixel locations of the original image) and wherein the method further comprises:
in response to the target pixel being within the foveal region, defining an associated pixel value of the target pixel to be equivalent to an associated pixel value of the source pixel (see, e.g., pars. 27, 33, 36, and 38-40 and FIGS. 4A-B and 5A-B of Kris, which teach that the sampling points in the ROI lie exactly on the pixel locations of the original image and no resampling is needed); and
in response to the target pixel being outside of the foveal region, defining an associated pixel value of the target pixel using a downsampling technique (see, e.g., pars. 32-33 and 38-40 and FIGS. 4A and 5A of Kris, which teach that sampling locations outside the ROI are more sparsely spread than the original pixel locations).
For claim 3, Kris in view of Berkovich teaches that the downsampling technique comprising a filtering technique (see, e.g., pars. 30, 32 and 36), the filtering technique comprising any of:
linear filtering comprising defining the associated pixel value of the target pixel to be an average of pixel values associated with source pixels mapped to the target pixel (see, e.g., pars. 30, 32 and 36 of Kris, which teach using the linear spacing and bilinear interpolation),
defining the associated pixel value of the target pixel to be a pixel value associated with a single source pixel mapped to the target pixel,
defining the associated pixel value of the target pixel to be a sum of source pixels mapped to the target pixel,
nearest neighbor filtering,
anisotropic filtering,
Lanczos filtering.
For claim 4, Kris in view of Berkovich teaches comprising encoding the target image to produce an encoded target image (see, e.g., pars. 24 and 33 of Kris, which teach encoding the downsampled image to generate a bitstream) and sending the encoded target image to a head-mounted device for display (see, e.g., pars. 34, 104-106 and 109, which teach sending the encoded downsampled image to a display, e.g., a display of a head mounted display device).
For claim 5, Kris in view of Berkovich teaches that the foveal region is defined using at least one characteristic of a device by which the target image is to be decompressed and displayed (see, e.g., par. 31 of Kris, which teaches that the ROI parameters are determined based on the threshold that is based on the screen size).
For claim 6, Kris in view of Berkovich teaches that the target image is for decompression and display on a display of a head-mounted device (see, e.g., pars. 34-35, 79, 104-106, and 109 of Kris, which teach sending the encoded downsampled image to be decoded/decompressed and displayed on a display, e.g., a display of a head mounted display device), and
wherein the foveal region is defined using at least one of (the examiner interprets the following conditions disjunctively):
a position of the display of the head-mounted device (see, e.g., pars. 99 and 104-106 and FIGS. 8A and B, which teach that the ROI is defined using the gaze direction with respect to the position of the display screen on the user’s head) and
a lens type of at least one lens of the head-mounted device.
For claim 7, Kris in view of Berkovich teaches that the foveal region is defined using at least one of:
a gaze direction of a user of a device by which the target image is to be decompressed and displayed (see, e.g., pars. 37 and 94-106 and FIGS. 8A-B of Kris, which teach that the ROI is determined based on the gaze tracking system).
For claim 8, Kris in view of Berkovich teaches that the foveal region is one of:
predefined prior to the receiving of the source image (see, e.g., par. 26 of Kris, which teaches determining the ROI parameters based on the predefined factors such as the required bit rate of the compressed image and the degree of quality loss outside the ROI) and
dynamically defined by the method (see, e.g., pars. 24-31 of Kris, which teach determining the ROI parameters based on the received frames).
For claim 9, Kris in view of Berkovich teaches that the distortion function is separable along a vertical and a horizontal dimension of the source and target images (see, e.g., pars. 33 and 39 and FIGS. 4A-B and 5A-B of Kris, which teach having different sampling sparsity between the axes), and
wherein the mapping of a source pixel of the source image to a target pixel of a target image using the distortion function comprises determining a target pixel by any of (the examiner interprets the following conditions disjunctively):
determining a horizontal position of the target pixel in the target image independently of determining a vertical position of the target pixel in the target image by applying the horizontal part of the separable distortion function to a horizontal position of the source pixel in the source image (see, e.g., pars. 32-33 of Kris, which teach changing the sample density along each axis so that a density along one axis is sparser than the other),
determining a vertical position of the target pixel in the target image independently of determining a horizontal position of the target pixel in the target image by applying the vertical part of the separable distortion function to a vertical position of the source pixel in the source image (see, e.g., pars. 32-33 of Kris, which teach changing the sample density along each axis so that a density along one axis is sparser than the other).
For claim 10, Kris in view of Berkovich teaches that the foveal region is a rectangular region of pixels (see, e.g., pars. 24-25 and FIGS. 3A-B of Kris).
For claim 11, Kris in view of Berkovich teaches that the more than one-to-one source-to-target pixel mapping outside of the foveal region is a linear mapping (see, e.g., pars. 30-32 and 36, and FIGS. 4A-B of Kris, which teach using a linear mapping/spacing for sampling pixels) .
For claim 12, Kris in view of Berkovich teaches that the more than one-to-one source-to-target pixel mapping outside of the foveal region is a quadratic mapping (see, e.g., pars. 38-40 and FIGS. 5A-B of Kris, which teach using a quadradic functions for sampling).
For claim 13, Kris in view of Berkovich teaches that a slope of the quadratic mapping matches a slope of the one-to-one source-to-target pixel mapping of the foveal region, at a pixel located at a boundary between the foveal region and the outside of the foveal region (see, e.g., pars. 38-40 and FIGS. 5A-B of Kris, which teach that the sampling locations are more closely tied to the original pixel locations as the samples are closer to the ROI, and in FIGS. 5A-B, the sampling points in the corner of the ROI, i.e., those in the boundary, correspond to the original pixel locations).
For claim 14, Kris as applied discloses a method for compressing a source image for sending to a head-mounted device for decompression and display, the method comprising:
receiving the source image, the source image having a source resolution and comprising at least one pixel with at least one associated pixel value (see, e.g., pars. 24-25 and FIGS. 1 and 3A, which teach that the received input/original image having a w0xh0 resolution/size);
mapping a source pixel of the source image to a target pixel of a target image using a distortion function, the target image having a lower resolution than the source resolution (see, e.g., pars. 24-33 and FIGS. 1, 3A and 4A, which teach sampling pixels of the original image to be the pixels of a downsampled image using a downsampling function),
wherein the distortion function defines a mapping (see, e.g., pars. 24-33 and FIGS. 1, 3A and 4A, which teach sampling pixels of the original image to be the pixels of a downsampled image using a downsampling function), the mapping comprising a one-to-one source-to-target pixel mapping within a foveal region (see, e.g., pars. 27 and 31-33 and FIG. 4A, which teach keeping the down sampling ratio in the ROI as 1 so that the sampling points lie exactly on the pixel locations of the original image and no resampling is needed), and the mapping comprising a more than one-to-one source-to-target pixel mapping outside of the foveal region (see, e.g., pars. 25-26 and 31-33 and FIG. 4A, which teach that the image is sampled more sparsely sampled outside the ROI), and wherein the foveal region is a defined area of pixels (see, e.g., pars. 25-27 and 31 and FIG. 3A, which tech defining a rectangular area with threshold height and width as the ROI);
in response to the target pixel being within the foveal region, defining an associated pixel value of the target pixel to be equivalent to an associated pixel value of the source pixel (see, e.g., pars. 27, 33, 36, and 38-40 and FIGS. 4A and 5A, which teach that the sampling points in the ROI lie exactly on the pixel locations of the original image and no resampling is needed);
in response to the target pixel being outside of the foveal region, defining an associated pixel value of the target pixel using a downsampling technique (see, e.g., pars. 32-33 and 38-40 and FIGS. 4A and 5A, which teach that sampling locations outside the ROI are more sparsely spread than the original pixel locations);
encoding the target image using a hardware encoding unit to produce an encoded target image (see, e.g., pars. 24 and 33, which teach encoding the downsampled image to generate a bitstream); and
sending the encoded target image to the head-mounted device (see, e.g., pars. 34 and 109, which teach sending the encoded data to a display, e.g., a display of a head mounted display device).
While Kris as applied does not explicitly teach, Berkovich in the analogous art teaches that the foveal region is defined using one or more of: a characteristic of a network via which the target image is to be transmitted, an attention of a user of a device by which the target image is to be decompressed and displayed (see, e.g., par. 13 of Berkovich teaches that the foveated sensing may be implemented based on factors such as user state including user attention), or an importance factor of an element of the source image.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kris to define the foveal region using additional factor taught by Berkovich because doing so would improve the use of system resource in an artificial reality system such as an HMD (see, e.g., par. 13 of Berkovich).
For claim 15, Kris in view of Berkovich teaches that the method at least partially carried out using hardware logic (see, e.g., pars. 80 and 116 of Kris, which teach using the hardware decoder and network interface).
For claim 16, Kris as applied discloses a method for decompressing a compressed image, comprising:
receiving the compressed image, the compressed image having a target resolution (see, e.g., pars. 34 and 76-77 and FIGS. 2 and 7, which teach receiving the compressed/encoded, downsampled image); and
mapping a target pixel of the compressed image to a source pixel of a source image using a distortion function (see, e.g., pars. 34-35 and FIGS. 3B and 4B, which teach mapping pixels of the decoded image to the pixels of the original image using an upsampling function), the source image having a higher resolution than the target resolution (see, e.g., pars. 34-35, which teach that the original size is reached by upsampling the decoded image),
wherein the distortion function defines a mapping (see, e.g., pars. 26-31 and 35-39 and FIGS. 3B and 4B, which teach using an overall downsampling ratio including ratios for the ROI and regions outside the ROI), the mapping comprising a one-to-one target-to-source pixel mapping within a foveal region (see, e.g., pars. 27, 33 and 36-39 and FIGS. 4B and 5B, which teach keeping the down sampling ratio in the ROI as 1 so that the sampling points lie exactly on the pixel locations of the original image and no resampling is needed), and the mapping comprising one-to-more-than-one target-to-source pixel mapping outside of the foveal region (see, e.g., pars. 25-26 and 35-39 and FIGS. 4B and 5B, which teach that the sampling density is higher for the background compared to the ROI during upsampling),
wherein the foveal region is a defined area of pixels (see, e.g., pars. 25-27 and 31 and FIGS. 3A-B, 4A-B and 5A-B, which tech defining a rectangular area with threshold height and width as the ROI).
While Kris as applied does not explicitly teach, Berkovich in the analogous art teaches that the foveal region is defined using one or more of: a characteristic of a network via which the target image is to be transmitted, an attention of a user of a device by which the target image is to be decompressed and displayed (see, e.g., par. 13 of Berkovich teaches that the foveated sensing may be implemented based on factors such as user state including user attention), or an importance factor of an element of the source image.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Kris to define the foveal region using additional factor taught by Berkovich because doing so would improve the use of system resource in an artificial reality system such as an HMD (see, e.g., par. 13 of Berkovich).
For claim 17, Kris in view of Berkovich teaches that the compressed image comprises at least one pixel with at least one associated pixel value (see, e.g., pars. 35-40 and FIGS. 4B and 5B of Kris), and wherein the method further comprises:
in response to the source pixel being within the foveal region, defining an associated pixel value of the source pixel to be equivalent to an associated pixel value of the target pixel (see, e.g., pars. 35-40 and FIGS. 4B and 5B of Kris, which teach upsampling the decoded downsampled image, wherein the associated pixel values lie in the original pixel locations); and
in response to the source pixel being outside of the foveal region, defining an associated pixel value of the source pixel using an upsampling technique (see, e.g., pars. 35-40 and FIGS. 4B and 5B of Kris, which teach upsampling the decoded downsampled image, wherein the background is upsampled with a higher density than the ROI).
For claim 18, Kris in view of Berkovich teaches that the upsampling technique comprising a filtering technique (see, e.g., pars. 30, 32 and 36 and FIG. 4B of Kris), the filtering technique comprising any of (the examiner interprets the following conditions disjunctively):
linear filtering, defining the associated pixel value of the source pixel to be an average of pixel values associated with pixels within a defined distance of the target pixel in the compressed image (see, e.g., pars. 30, 32 and 36 and FIG. 4B of Kris, which teach using the linear spacing and bilinear interpolation),
defining the associated pixel value of the source pixel to be an associated pixel value of the target pixel,
nearest-neighbor filtering,
defining the associated pixel value of the source pixel to be an average of pixel values associated with pixels within a defined distance of the source pixel in the source image wherein a pixel of the source image used for the upsampling technique without an associated value has an associated value defined for the upsampling technique to be an associated pixel value of a pixel of the compressed image that is mapped to the pixel of the source image,
anisotropic filtering,
Lanczos filtering.
For claim 19, Kris in view of Berkovich teaches that the method at least partially carried out using hardware logic (see, e.g., pars. 80 and 116 of Kris, which teach using the hardware decoder and network interface).
For claim 20, Kris in view of Berkovich teaches that the compressed image is encoded (see, e.g., pars. 33-34 and FIGS. 2 and 3B of Kris, which teach that the downsampled image is encoded), the method further comprising decoding the compressed image prior to the mapping (see, e.g., pars. 33-34 and FIGS. 2 and 3B, which teach that the encoded image is decoded before upsampling), and the method further comprising displaying the source image on a display of a head-mounted device (see, e.g., pars. 34 and 109 and FIG. 2 of Kris, which teach that the upsampled, decoded image is displayed on a display, e.g., a display of a head mounted display device).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WOO RHIM whose telephone number is (571)272-6560. The examiner can normally be reached Mon - Fri 9:30 am - 6:00 pm et.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Henok Shiferaw can be reached at 571-272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WOO C RHIM/Examiner, Art Unit 2676
Prosecution Insights