DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Prior arts cited in this office action:
Fitzgerald et al. (US 20200280739 A1, hereinafter “Fitzgerald”)
Strandborg et al. (US 20210248941 A1, hereinafter “Strandborg”)
Lakshminarayanan (US 20240212192 A1, hereinafter “Lakshminarayanan”)
Ishita et al. (JP 2014160380 A1, hereinafter “Ishita”)
Response to Arguments
Applicant's arguments filed on 04/28/2026 have been fully considered but they are not persuasive.
Applicant’s Arguments/Remarks: the present claims include both an importance factor calculation and an encoding process where the importance factor is based on both an extent of change between corresponding areas of a previous image and a given image, and a distance of the given area from the gaze location, and the encoding uses previous encoded data when the importance factor is below a threshold, instead of encoding the given area. Furthermore, the present claims clarify that when the extent of change increases the importance factor increases, and when the distance increases the importance factor decreases. Fitzgerald fails to disclose or suggest these features.
While Strandborg discloses amplifying or attenuating brightness differences in color pixels, depending on whether the color pixels are inside or outside a region of interest, Strandborg also lacks any disclosure related to encoding using previous encoded data of the corresponding area, when the importance factor is below a threshold, instead of encoding the given area of the given image into the encoded image data.
Examiner’s Response: examiner disagrees with applicant assertion above that the combination of the cited prior arts does not teach or suggest applicant invention as claimed and argued above. Applicant tries to argue the references separately while the rejection is based on the combination of the references. As cited in the previous action Fitzgerald teaches Thus, this provides a mechanism for compressing and subsequently decompressing the image, with the compression being controlled based on the location of an array of pixels relative to a defined point. Specifically this allows a degree of compression to be selected based on the position of the array of pixels, so that, less compression can be used in more important (important factor based on distance) parts of an image, such as in a region proximate the point of gaze, whilst greater compression (when distance increases the importance decreases) is used in other areas of the image, such as further from the point of gaze, for example in the users peripheral field of view (Fitzgerald [0209], [0212], [0286]). Also, Fitzgerald further teaches determining the extent of change from one image frame to the next such that the amount of pixels or pixel areas that need to be replaced is determined. The change or the amount of change dictate the compression to be applied such that the system is optimized (Fitzgerald [0104], [0130]
And a pointed out by applicant “Strandborg discloses amplifying or attenuating brightness differences in color pixels, depending on whether the color pixels are inside or outside a region of interest” and that “Optionally, the distance factor decreases with an increase in the distance. In other words, the distance factor is inversely related to the distance of the given pixel from the gaze point on the image plane. Optionally, the function is selected, by the at least one processor, in a manner that upon updating the initial second output value using the distance factor, a difference between the second output value and the first output value of the given colour component (that is greater than or equal to the first threshold difference) is: [0085] amplified for pixels within the region of interest around the gaze point in the second output image frame, and [0086] attenuated (progressively) for pixels outside the region of interest in the second output image frame” (Strandborg [0084]). Where that the change around the gaze point or the second output value and the first output values is greater than a threshold the pixels around the gaze point are amplified, in other words, more importance id given when the change is more to those pixels.
Furthermore, applicant is reminded that the test for obviousness is not whether the features of a secondary reference may be bodily incorporated into the structure of the primary reference; nor is it that the claimed invention must be expressly suggested in any one or all of the references. Rather, the test is what the combined teachings of the references would have suggested to those of ordinary skill in the art. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981). In this case using both distance and amount of change to determine the importance of an area and how to process that area.
Claim 8 contains similar limitation as claim 1 and is therefore rejected on the same ground as claim 1 above.
Claim 2-7, 9-14 depend at least on either claim 1 or 8 and are therefore not allowable for the same reason above.
Drawings
The drawings are objected to under 37 CFR 1.83(a). The drawings must show every feature of the invention specified in the claims. Therefore, the boxes in the figures must be properly labeled and corresponding features must be shown or the feature(s) canceled from the claim(s). No new matter should be entered. For example, the box 106, 102, 202 and so on in figures 1 and 2 must describe what it is with corresponding label corresponding to the specification not just the number (106, 102 or 202).
Corrected drawing sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office action to avoid abandonment of the application. Any amended replacement drawing sheet should include all of the figures appearing on the immediate prior version of the sheet, even if only one figure is being amended. The figure or figure number of an amended drawing should not be labeled as “amended.” If a drawing figure is to be canceled, the appropriate figure must be removed from the replacement sheet, and where necessary, the remaining figures must be renumbered and appropriate changes made to the brief description of the several views of the drawings for consistency. Additional replacement sheets may be necessary to show the renumbering of the remaining figures. Each drawing sheet submitted after the filing date of an application must be labeled in the top margin as either “Replacement Sheet” or “New Sheet” pursuant to 37 CFR 1.121(d). If the changes are not accepted by the examiner, the applicant will be notified and informed of any required corrective action in the next Office action. The objection to the drawings will not be held in abeyance.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 8 and 15 are rejected under 35 U.S.C. 103 as being unpatentable over Fitzgerald et al. (US 20200280739 A1, hereinafter “Fitzgerald”) in view of Strandborg et al. (US 20210248941 A1, hereinafter “Strandborg”) in view of Ishita et al. (JP 2014160380 A1, hereinafter “Ishita”).
Regarding claims 1, 8 and 15:
Fitzgerald teaches a computer-implemented method (Fitzgerald [0001], [0073], [0094], where Fitzgerald discloses a computer implemented method and apparatus) comprising:
identifying a gaze location in a given image, based on a given gaze direction (Fitzgerald [0028], where Fitzgerald teaches in one embodiment the method includes: determining a user gaze including at least one of: a direction of gaze; and, a depth of gaze; using the user gaze to at least one of: compress image data; and, decompress the compressed image data);
dividing the given image into a plurality of areas (Fitzgerald [0132], [0163], [00170], [0212], where Fitzgerald discloses the image is divided into a plurality of regions or areas of interest having arbitrary shape);
for a given area of the given image, identifying a corresponding area in at least one previous image (Fitzgerald [0026], [0104], [0130], claim 15, where Fitzgerald teaches In one embodiment the method includes: determining a change in display device pose from display of a previous image using at least one of: movement data; and, pose data and previous pose data; and, using the change in display device pose to at least one of: compress image data; and, decompress the compressed image data);
determining an extent of change between the corresponding area of the at least one previous image and the given area of the given image (Fitzgerald [0026], [0104], [0130]- [0132] claim 15, where Fitzgerald teaches the appearance of individual objects within an image may be unchanged between successive images, with only the position varying based on movement of the display device. Accordingly, in this example, it is possible to simply replace portions of an image with part of previous image. The display device can then retrieve image data from the previous image and substitute this into the current image, vastly reducing the amount of image data that needs to be transferred without resulting in any loss in information;
and
encoding the given image into encoded image data, wherein when the importance factor for the given area is smaller than a first predefined threshold, the step of encoding comprises re-using previous encoded data of the corresponding area, instead of encoding the given area of the given image into the encoded image data image (Fitzgerald [0026], [0104], [0130]- [0132] claim 15, where Fitzgerald teaches The display device can then retrieve image data from the previous image and substitute this into the current image, vastly reducing the amount of image data that needs to be transferred without resulting in any loss in information. This could be calculated based on the display device movement, and/or could be achieved through code substitution, for example by replacing part of an image with a reference to part of a previous image, with the reference being transmitted as part of the image data. The reference could be of any appropriate form, but in one example is a code or similar that refers to a region within the earlier image. This could include a specific region of pixels, such as one or more pixel arrays, or could refer to a region defined by a boundary, as will become apparent from the remaining description).
Fitzgerald fails to teaches explicitly calculating an importance factor for the given area of the given image, based on the determined extent of change and a distance of the given area from the gaze location, such that when the extent of change increases the importance factor increases, and when the distance increases the importance factor decreases;
However, Fitzgerald teaches Thus, this provides a mechanism for compressing and subsequently decompressing the image, with the compression being controlled based on the location of an array of pixels relative to a defined point. Specifically this allows a degree of compression to be selected based on the position of the array of pixels, so that, less compression can be used in more important (important factor based on distance) parts of an image, such as in a region proximate the point of gaze, whilst greater compression (when distance increases the importance decreases) is used in other areas of the image, such as further from the point of gaze, for example in the users peripheral field of view (Fitzgerald [0209], [0212], [0286]). Furthermore, Strandborg teaches it will be appreciated that the color reproduction capabilities are significantly improved when generating the output image frames based on the gaze direction of user’s eye. This is attributed to the fact that color reproduction for different output regions of the output image frames, such as the first output region and the second output region, is performed differently in an optimized manner, using the distance factor (Strandborg [0078]-[0089], [0138], [0148]-[0151]). And Ishita teaches the local pattern change amount represents the change amount of the pixel value between surrounding pixels. Such a region having a large amount of change from the surrounding pixels is a region that tends to be noticed because it is a region having a high frequency component. Accordingly, a higher value is assigned as the degree of importance as the amount of change is larger… On the other hand, the global pattern similarity means the amount of change between partial areas composed of a set of a plurality of pixels. When the similarity between the partial areas is high, the importance is low because it is considered to be an area where a similar texture such as grassland or grass continues. On the other hand, when the degree of similarity is low, it is considered that the area is a mixture of different textures, and therefore, the degree of importance is high (Ishita [0048]-[0052], figs. 5-8).
Therefore, taking the teachings of Fitzgerald and Strandborg and Ishita as whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to determine an importance factor for the given area of the given image, based on the determined extent of change and a distance of the given area from the gaze location, such that when the extent of change increases the importance factor increases, and when the distance increases the importance factor decreases, in order to determine the type of encoding to apply such that the best image possible can be obtained that provides the necessary detail information while minimizing the amount of data that needs to be processed by the system.
Claims 2-14 are rejected under 35 U.S.C. 103 as being unpatentable over Fitzgerald et al. (US 20200280739 A1, hereinafter “Fitzgerald”) in view of Strandborg et al. (US 20210248941 A1, hereinafter “Strandborg”), in view of Ishita et al. (JP 2014160380 A1, hereinafter “Ishita”) and in view of Lakshminarayanan (US 20240212192 A1, hereinafter “Lakshminarayanan”).
Regarding claims 2 and 9:
Fitzgerald in view of Strandborg and in view of Ishita fails to explicitly teaches further comprising reprojecting the at least one previous image from a corresponding previous pose to a given pose, prior to identifying the corresponding area in the at least one previous image, the at least one previous image and the given image being rendered according to the corresponding previous pose and the given pose, respectively.
However, Lakshminarayanan teaches A “shift per requirement” (e.g., in terms of a relative position within an image frame) may be applied to first and second dynamic objects D1 and D2 during respective object frame periods. As described with reference to FIG. 3, for example, control circuit 110 may include a sub-image shifter 370 configured to determine whether and the extent to which a positional shift within an image frame should be applied to a first dynamic object D1 or to a second dynamic object D2. The shift determination may be based, at least in part, on respective motion vectors determined for dynamic objects D1 and D2, as explained with reference to FIG. 1. The motion vector may be based on, for example, a movement vector of object D1 or D2, any directional change of the gaze of the viewer's pupil(s), or any movement vector of the viewer's head or entire body. Movement of the viewer's head or body may be determined, for example, based on detected movement of control circuit 110. In some systems, a respective movement vector of a projected object may be encoded as part of the video data for object D1 or D2, or may be derived by control circuit 110 by comparing positions of a given image in one image frame relative to another (Lakshminarayanan [0023] [0127], [0137]).
Therefore, taking the teachings of Fitzgerald, Strandborg, Ishita and Lakshminarayanan as a whole, it would have been obvious to one of ordinary skill in the art before the effective filing date of the application to perform projection of the two images or image frames to determine the amount of different between the images, such as difference between corresponding regions, since using projection in this manner is well-known technique that when used provide a good indication and highlight the difference between images.
Regarding claims 3 and 10:
Fitzgerald in view of Strandborg, in view of Ishita and in view of Lakshminarayanan teaches wherein the at least one previous image comprises a plurality of previous images, the method further comprising tracking changes in the given area across a sequence of images, said sequence comprising the plurality of previous images and the given image, wherein the importance factor is calculated for the given area of the given image, further based on at least one of: an extent of the tracked changes across the sequence of images, a rate with which the changes have occurred across the sequence of images (Fitzgerald [0020], [0043], [0127]; Strandborg [0075], Lakshminarayanan [0022]-[0023] [0127], [0137], where Fitzgerald in view of Strandborg and in view of Lakshminarayanan teaches In one embodiment the image forms part of a sequence of images, and wherein the method includes using respective display data to compress and decompress at least one of: image data for a sub-sequence of one or more images; and, image data for each image).
Regarding claims 4 and 11:
Fitzgerald in view of Strandborg, in view of Ishita and in view of Lakshminarayanan teaches further comprising when the importance factor for the given area is greater than or equal to the first predefined threshold, but smaller than a second predefined threshold, interpolating between original values of pixels of the given area of the given image and values of corresponding pixels of the corresponding area of the at least one previous image, based on the importance factor calculated for the given area, to generate interpolated values for the pixels of the given area, and encoding the interpolated values into the encoded image data (Lakshminarayanan [0022]-[0023] , [0090], [0118], [0127], [0137], where intermediate position, for example can be interpreted as performing interpolation by one of ordinary skill in the art).
Regarding claims 5 and 12:
Fitzgerald in view of Strandborg, in view of Ishita and in view of Lakshminarayanan teaches further comprising when the importance factor for the given area is greater than the second predefined threshold, encoding original values of the pixels of the given area into the encoded image data (Fitzgerald [0026], [0104], [0130]- [0132] claim 15; Strandborg [0012]; Lakshminarayanan [0022]-[0023] , [0090], [0118], [0127], [0137]).
Regarding claims 6 and 13:
Fitzgerald in view of Strandborg, in view of Ishita and in view of Lakshminarayanan teaches further comprising attaching, with the given image, metainformation indicative of at least one of:
areas of the given image for which previous encoded data of their corresponding areas of the at least one previous image are to be re-used,
positions of the corresponding areas of the at least one previous image,
relative positions of the corresponding areas of the at least one previous image with respect to the areas of the given image,
respective rotation to be applied to the corresponding areas, respective scaling to be applied to the corresponding areas (Fitzgerald [0034], [0046], [0057], [0146], [0293]; Lakshminarayanan [0022]-[0023] , [0090], [0118], [0127], [0137]).
Regarding claims 7 and 14:
Fitzgerald in view of Strandborg, in view of Ishita and in view of Lakshminarayanan teaches wherein for each area of the given image whose importance factor is smaller than the first predefined threshold, the encoded image data comprises a reference to previous encoded data of a corresponding area of the at least one previous image that is to be re-used for said area of the given image (Fitzgerald [0026], [0104], [0130]- [0132] claim 15; Strandborg [0012] Lakshminarayanan [0022]-[0023], [0090], [0118], [0127], [0137]).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WEDNEL CADEAU whose telephone number is (571)270-7843. The examiner can normally be reached Mon-Fri 9:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chieh Fan can be reached at 571-272-3042. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WEDNEL CADEAU/Primary Examiner, Art Unit 2632 June 12, 2026
Prosecution Insights