DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5-9, 12-13, 15-20, and 24 are rejected under 35 U.S.C. 103 as being unpatentable over Bosse (WO 2020/069976) in view of Oh et al. (US Patent No. 12,003,691.
With reference to claims 1, 16, and 20, Bosse discloses an apparatus for graphics processing (14) (see pg. 7, ln. 29-pg. 8, ln 6; Fig. 4), comprising:
a memory (see pg. 8, ln 35-pg. 9, ln 2); and
a processor (20) coupled to the memory and, based on information stored in the memory (see pg. 8, lns. 15-23; Fig. 4), the processor is configured to:
receive/transmit, over a network, a first indication of a gaze of a user, wherein the first indication indicates a first ROI (ROI) (8) for a frame (see pg. 8, lns. 1-23; Figs. 1-2);
compute, based on the a first ROI (8), an importance map comprising a first confidence value associated with the first ROI (see pg. 13, lns. 5-17; pg. 17, lns. 30-33);
encode, based on the computed importance map, the frame such that the first ROI (8) of the frame comprises a first quality (see pg. 2, ln. 27-pg. 3, ln 9); and
output a second indication of the encoded set of regions of the frame (see pg. 19, lns 5-19;).
While disclosing receiving a first indication and outputting a second indication as recited, Bosse fails to specifically recite that the first indication indicates a first and second ROI as recited.
Oh discloses a HMD which identifies a ROI based on the user’s viewpoint (see abstract; column 2, lines 65-column 3, line 3; column 5, lines 7-31; Figs. 1-4), wherein a processor receives a first indication of a gaze (502) of a user (504) (see column 5, lines 55-67), wherein the first indication indicates a first region (406) of interest and a second region (408) of interest for a frame (304), compute, and importance map comprising a first confidence value associated with the first ROI and a second confidence value associated with the second ROI, wherein the first confidence value is higher than the second confidence value (in teaching confidence value associated with the user’s gaze towards the ROI; see column 9, line 31-column 10, line 6); wherein the first ROI (406) of the frame has a first quality, the second ROI of the frame comprises a second quality, and a third region of the frame comprises a third quality, wherein the first quality is higher than the second quality, wherein the second quality is higher than the third quality (in teaching two or more imaging cameras to be set to different focus levels at the same time, wherein “any suitable number of ROIs may be identified”; column 5, lines 46-48; column 9, line 50-column 10, line 27).
Therefore it would have been obvious to one of ordinary skill in the art to allow identifying the first and second region similar to that which is taught by Oh to be carried out in a system similar to that which is taught by Bosse to thereby allow focus adjustment of an object being viewed by the user (see Oh; column 1, lines 35-38).
Therefore it would have been obvious to one of ordinary skill in the art to allow identifying the first and second region similar to that which is taught by Oh to be carried out in a system similar to that which is taught by Bosse to thereby allow focus adjustment of an object being viewed by the user (see Oh; column 1, lines 35-38).
With reference to claim 3, Bosse and Oh disclose the apparatus of claim 1, wherein Bosse further discloses wherein, to encode the frame, the processor is configured to set a first bitrate for the first ROI of the frame and a second bitrate for the third region of the frame, wherein the first bitrate is higher than from the second bit rate (in teaching sub-predictors of the viewpoint (8); see pg. 8, ln. 15-pg. 9, ln. 2; Fig. 1); and encode the frame further based on the first bitrate and the second bitrate (see pg. 20, ln. 28-pg. 21, ln 27).
With reference to claim 5, Bosse and Oh disclose the apparatus of claim 1, wherein Bosse further discloses wherein the first indication of the gaze of the user comprises a set of predicted eye gaze poses of the user, or a set of confidence values for the set of predicted eye gaze poses (see column 9, line 31-column 10, line 6).
While disclosing eye gaze poses as recited, Bosse fails to specifically disclose the first and second ROI as recited.
Oh discloses wherein a first eye gaze pose of the set of eye gaze poses of the user corresponds with the first ROI and a second eye gaze pose of the set of eye gaze poses of the user corresponds with the second ROI (see column 5, lines 6-67; column 9, line 50-column 10, line 27; Figs. 3-4)
With reference to claim 6, Bosse and Oh disclose the apparatus of claim 5, wherein Bosse further discloses wherein, to compute the importance map, the processor is further configured to: normalize the set of confidence values based on the set of predicted eye gaze poses (see pg. 8, lns. 1-23); and compute the importance map based on the normalized set of confidence values (see pg. 12, ln. 13-pg. 13, ln. 17).
Oh further discloses encoding based on the first ROI and the second ROI of the gaze of the user (see column 5, lines 6-67; Figs. 3-4).
With reference to claim 7, Bosse and Oh disclose the apparatus of claim 6, wherein Bosse further discloses wherein, to compute the importance map, the processor is further configured to: sort the set of predicted eye gaze poses based on the normalized set of confidence values (see pg. 12, ln. 13-pg. 13, ln. 17).
Oh further discloses encoding based on the first ROI and the second ROI of the gaze of the user (see column 5, lines 6-67; Figs. 3-4).
With reference to claim 8, Bosse and Oh disclose the apparatus of claim 7, wherein Bosse further discloses wherein, to compute the importance map, the processor is further configured to: identify, based on (1) the sorted set of predicted eye gaze poses and (2) the normalized set of confidence values, the set of regions of the frame; and assign, based on the normalized set of confidence values, a set of importance values to the set of regions of the frame (in teaching confidence information of the predicted viewpoint location; see pg. 17, lns. 19-33), wherein, to encode the frame based on the computed importance map, the processor is configured to: encode the frame based on the assigned set of importance value (see pg. 20, ln. 28-pg. 21, ln 27).
Oh further discloses encoding based on the first ROI and the second ROI of the gaze of the user (see column 5, lines 6-67; Figs. 3-4).
With reference to claim 9, Bosse and Oh discloses the apparatus of claim 1, wherein Bosse discloses the importance map for encoding a frame (in teaching prediction of the position of the viewport (8); see pg. 2, ln. 27-pg. 3, ln 9; pg. 4, lns 5-18), however fails to disclose the first and second regions of interest as recited.
Oh further discloses and further discloses wherein, to compute, based on the first ROI and the second ROI of the gaze of the user (see column 5, lines 6-67; Figs. 3-4), and the processor is further configured to: determine that the first ROI and a fourth ROI of the frame (in teaching “any suitable number of ROIs may be identified”; column 5, lines 46-48; column 9, line 50-column 10, line 27) in the set of regions are within a threshold distance of one another (in teaching capturing of environmental image (600) including multiple regions of interest (602, 604); see column 7, line 4-9; Fig. 6); merge, based on the determination, the first ROI and the fourth ROI into a single region (see column 10, lines 28-58; Figs. 8-9); and assign, based on a first importance value of the first ROI and a second importance value of the fourth ROI, a single confidence value to the single ROI (see column 10, lines 24-40; Figs. 8-9).
With reference to claim 12, Bosse and Oh discloses the apparatus of claim 1, wherein Bosse further discloses wherein, the processor is further configured to: obtain a third indication comprising at least one of prior gaze information for the user (see pg. 12, lns., 9-11), animation elements associated with the frame, interactivity elements associated with the frame (in teaching gaming experience; pg. 9, lns. 24-27), or ROI (ROI) information associated with the frame (in teaching viewpoint (8); see Fig. 1), wherein to compute the importance map, the processor is configured to compute the importance map (out of sight area) further based on the third indication (see pg. 7, lns. 29-34; Fig. 1).
Oh further discloses encoding based on the first ROI and the second ROI of the gaze of the user (see column 5, lines 6-67; Figs. 3-4).
With reference to claim 13, Bosse and Oh discloses the apparatus of claim 1, wherein Bosse further discloses wherein the frame is associated with extended reality (XR) content of a device (see pg. 19, lns. 5-19), and wherein the device comprises one of a wearable display device, a headset, or a head-mounted display (HMD) (see pg. 7, lns, 29-34; Figs. 1-2).
With reference to claim 15, Bosse and Oh discloses the apparatus of claim 1, wherein Bosse further discloses wherein, the apparatus comprises a wireless communication device comprising at least one of a transceiver or an antenna coupled to the processor, wherein the processor is further configured to (see pg. 8, lns. 1-7; Fig. 2), wherein, the processor is further configured to: perform a negotiation (102) with a device to enable eye tracking and transmission, wherein, to receive via at least one of the transceiver or the antenna the first indication of the gaze, the processor is configured to receive the first indication of the gaze subsequent to the performed negotiation (see pg. 8, lns. 1-13, pg. 10, lns. 11-26; Figs. 1-3).
With reference to claim 17, Bosse discloses the apparatus of claim 16, and further discloses wherein, the first indication of the gaze of the user comprises at least one of a set of predicted eye gaze poses of the user and a set of confidence values for the set of predicted eye gaze poses (see pg. 8, lns. 15-23; pg. 9, lns. 19-23), wherein the processor is further configured to: generate, via a camera of the device, the set of predicted eye gaze poses of the user (see pg.9, lns. 19-23); obtain, via an extended reality (XR) runtime (see pg. 19, lns. 5-19), the set of confidence values for the set of predicted eye gaze poses, or estimate, based on confidence criteria, the set of confidence values for the set of predicted eye gaze poses (in teaching confidence information of the predicted viewpoint location; see pg. 17, lns. 19-33).
While disclosing predicted eye gaze poses of the user as recited, Bosse fails to disclose the first and second ROI as recited.
Oh further discloses wherein a first eye gaze pose of the set of eye gaze poses of the user corresponds with the first ROI and a second eye gaze pose of the set of eye gaze poses of the user corresponds with [the] second ROI (see column 5, lines 6-67; Figs. 3-4).
With reference to claim 18, Bosse and Oh discloses the apparatus of claim 17, wherein Bosse further discloses wherein, the confidence criteria comprise at least one of, a time difference between a first time instance corresponding to a transmission of the first indication and a second time instance corresponding to a display time for the frame, a set of head poses of the user, or an estimated velocity of a head of the user (pg. 11, ln. 32-pg. 12, ln. 11).
With reference to claim 19, Bosse and Oh discloses the apparatus of claim 16, wherein Bosse further discloses wherein, the bitstream is encoded, wherein to output the indication of the received bitstream, the processor is configured to: decode the bitstream to produce the frame (108); and transmit the frame for display on the display panel (112) (in teaching encoding, which inherently includes decoding; see pg. 8, lns. 30-35; Fig. 3).
With reference to claim 24, Bosse and Oh disclose the apparatus of claim 5, wherein Bosse further discloses wherein, to encode the frame, the processor is configured to: encode, based on the set of confidence values for the set of predicted eye gaze poses, each computed ROI of the frame with a quality that scales with a corresponding confidence value of the set of confidence values (in teaching confidence information of the predicted viewpoint location; see pg. 17, lns. 19-33)
Claims 10-11 are rejected under 35 U.S.C. 103 as being unpatentable over Bosse and Oh as applied to claim 8 above, and further in view of Mammou et al. (US Patent Publication No. 2017/0302918; hereafter Mammou).
With reference to claim 10, Bosse and Oh discloses the apparatus of claim 8, wherein Oh discloses the first ROI and the second ROI (see column 5, lines 6-67; Figs. 3-4), however fails to disclose CTUs as recited.
Mammou discloses a system, method, and apparatus for streaming virtual reality content (see abstract, 20, 29) and further discloses wherein, each of the set of regions is associated with one or more coding tree units (CTUs) (see paragraph 33).
Therefore it would have been obvious to one of ordinary skill in the art to allow the usage of a CTU similar to that which is taught by Mammou to be carried out in a system similar to that which is taught by Bosse and Oh to thereby improve the visual experience (see Mammou; paragraphs 2-3).
With reference to claim 11, Bosse, Oh, and Mammou discloses the apparatus of claim 10, wherein Bosse further discloses wherein, to identify the first ROI and the second ROI of the frame, the processor is further configured to identify a set of importance regions and a set of non-importance regions (out of sight region; see pg. 7, lns. 29-34), and wherein to assign the set of importance values, the processor is configured to assign a minimum importance value to each of the set of non-importance regions (in teaching processing video stream based on a predicted location of the viewpoint portion; see pg. 8, lns. 1-6).
Oh further discloses the first ROI and the second ROI of the frame, wherein the set of non-importance regions (i.e. trees) comprises the third region of the frame (see column 5, lines 6-67; Figs. 3-4).
Claim 21 is rejected under 35 U.S.C. 103 as being unpatentable over Bosse and Oh as applied to claim 5 above, and further in view of Itsumi et al. (US Patent Publication No. 2023/0343106; hereafter Itsumi).
With reference to claim 21, Bosse and Oh disclose the apparatus of claim 5, wherein Bosse discloses wherein, to compute, based on the first ROI and the second ROI of the gaze of the user (see pg. 8, lns. 1-23), the importance map for the encoding of the frame (see pg. 12, ln. 13-pg. 13, ln. 17), the processor is configured to: assign, based on the set of confidence values for the set of predicted eye gaze poses (in teaching confidence information of the predicted viewpoint location; see pg. 17, lns. 19-33).
Oh further discloses the first ROI and the second ROI of the gaze of the user (see column 5, lines 6-67; Figs. 3-4) as explained above.
Bosse and Oh fail to disclose a radius of the first and second ROI as recited.
Itsumi discloses a system rich improves image quality at determined ROI (see paragraphs 68, 78; Figs. 5-9) wherein the processor (200) is configured to assign, based on confidence values, a radius for each computed ROI of the frame (see paragraphs 71-72; Figs. 8, 13), wherein the first radius of the first ROI is larger than a second radius of the second ROI (see paragraphs 72-73).
Therefore it would have been obvious to one of ordinary skill in the art to allow forming a first and second radius of the regions of interest similar to that which is taught by Itsumi to be carried out in a system similar to that which is taught by Bosse and Oh to thereby determine a region to be subjected to predetermined image processing (see Itsumi; paragraph 62).
Claim 23 is rejected under 35 U.S.C. 103 as being unpatentable over Bosse and Oh as applied to claim 1 above, and further in view of Ollila (US Patent Publication No. 2022/0147140).
With reference to claim 23, Bosse and Oh discloses the apparatus of claim 1, however fail to disclose PPD values as recite.
Ollila discloses an encoder for encoding images and a processor configured to receive information indicative of a gaze direction of a user (604) and divide the input image into a plurality of portions (606) surrounding the gaze location (X) (see abstract; paragraph 138; Figs. 3-6), wherein the first quality (502) comprises a first pixel per degree (PPD) value of the frame, wherein the second quality (504) comprises a second PPD value of the frame (see paragraphs 60, 66, 98-99; Figs. 4-6).
Therefore it would have been obvious to one of ordinary skill in the art to allow the usage of PPD values similar to that which is taught by Ollila to be carried out in a system similar to that which is taught by Bosse and Oh to thereby provide acceptable level of image quality (see Ollila; paragraph 4).
Response to Arguments
Applicant's arguments filed 01/26/2026 have been fully considered but they are not persuasive. With reference to claims 1, 16, and 20, the applicant argues that Bosse fails to teach computing based on a first and second ROI an importance map comprising a confidence value associated with the first ROI and a confidence value associated with the second ROI, and encode based on the importance map, wherein a first, second, and third ROI have different qualities as recited. The applicant further states that Oh fails to remedy the defect of Bosse with respect to the claims. The examiner finds that the amendment made to the claims fail to overcome the previous rejection. As discussed in the interview the claims fail to recite the invention as described with reference to Fig. 7, and therefore fails to overcome the previous rejection as explained in the interview. Specifically the applicant argues that Bosse fails to disclose the first and second ROI and importance map comprising confidence values associated therewith. The examiner finds that Bosse discloses confidence values in teaching confidence value associated with the user’s gaze towards the ROI (see column 9, line 31-column 10, line 6). Bosse teaches that based on the confidence values the quality of the ROI is change to be higher that other ROI. Further, Oh discloses multiple ROI having different quality values as recited. Mammou and Itsumi are not cited for teaching features of claim 1, 16, and 20 as argued. Therefore the examiner finds that the combination of references disclose the invention as recited. The examiner strongly suggest defining the invention of the ROI being of the same interest as described with reference to Fig. 7 as discussed in the interview.
Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
PRICE et al. (US2019/0058859) discloses a HMD (300) which identifies a ROI based on the user’s viewpoint (see paragraphs 63-66; Figs. 1-4), wherein a processor receives a first indication of a gaze of a user (FOV, 420) (see paragraphs 67-68; Figs. 4-5), wherein the first indication indicates a first ROI (412) and a second ROI (414) for a frame (500) (see paragraphs 68-69; Fig. 4-6).
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ALECIA DIANE ENGLISH whose telephone number is (571)270-1595. The examiner can normally be reached Mon.-Fri. 7:00am-3:00am.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Boddie can be reached on 571-272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADE/Examiner, Art Unit 2625
/WILLIAM BODDIE/Supervisory Patent Examiner, Art Unit 2625
Prosecution Insights