DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 03/26/2026 has been entered.
Response to Arguments
Applicant's arguments filed 3/26/2026 have been fully considered.
Regarding to claim 1, the applicant argues the cited arts fail to teach or suggest “receiving, by the XR device, media streams received respectively from a plurality of applications running in the XR device, each of the plurality of applications providing different services; determining, by the XR device, perceptual quality scores respectively for the media streams received from the plurality of applications, wherein the perceptual quality scores represent media qualities perceived by a user while using the plurality of applications in the XR device; based on the perceptual quality scores, determining, by the XR device, at least one candidate application among the plurality of applications with a different media quality from remaining applications among the plurality of applications; determining, by the XR device, a target perceptual quality score for at least one media stream of the at least one candidate application based on the perceptual quality scores, a capability of the XR device, and a plurality of network parameters; and harmonizing, by the XR device, at least one media parameter of the at least one media stream of the at least one candidate application based on the target perceptual quality score.” as presently recited in claim 1. The arguments have been fully considered. The argument according “each of the plurality of applications providing different services “ is persuasive. Therefore, the 35 U.S.C 103 rejection has been withdrawn. However, upon further consideration, new grounds of rejection are made in a newly applied art. The argument according rest claim limitations is not persuasive. The examiner cannot concur with the applicant for following reasons:
Li discloses “receiving, by the XR device, media streams received respectively from a plurality of applications running in the XR device”. For example, in paragraph [0098], Li teaches video and audio are not the same, and they are distinct, separate, and complementary media formats; Li further teaches video and audio quality becomes poor; Li further more teaches video and audio are two different streams from video application and audio application; In addition, Li teaches when network transmission quality is relatively poor, the VR multimedia may be frozen or video and audio quality becomes poor. In paragraph [0100], Li teaches the video image and audio of the VR multimedia are two different streams from two applications. In paragraph [0109], Li teaches video quality and audio quality of the VR multimedia. In Fig. 1 and paragraph [0113], Li teaches the server 102 obtains, based on the received VR multimedia obtaining request, VR multimedia, i.e. media stream, and related information required for playing the VR multimedia, and send the VR multimedia and the related information to the terminal 101. In paragraph [0117], Li teaches the terminal 101 receives the VR multimedia, i.e. media stream, and the interactive action of the user.
Li discloses “determining, by the XR device, perceptual quality scores respectively for the media streams received from the plurality of applications”. For example, in paragraph [0098], Li teaches video and audio are not the same; they are distinct, separate, and complementary media formats; Li further teaches video and audio quality becomes poor; Li further more teaches video and audio are two different streams from video application and audio application; In addition, Li teaches when network transmission quality is relatively poor, the VR multimedia may be frozen or video and audio quality becomes poor. In paragraph [0100], Li teaches the video image and audio of the VR multimedia are two different streams from two applications. In Fig. 1 and paragraph [0113], Li teaches determining experience quality of VR multimedia. In paragraph [0127], Li teaches determining experience quality of VR multimedia. In paragraph [0129], Li teaches calculating an experience score corresponding to each of at least two perceptual dimensions. In paragraph [0134], Li teaches determining a fidelity experience score, an enjoyment experience score, and an interaction experience score of the VR multimedia. In paragraph [0268], Li teaches determining the fidelity experience score, the enjoyment experience score, and the interaction score. In paragraph [0269], Li teaches calculating and outputting a fidelity experience score, an enjoyment experience score, and an interaction score of the VR multimedia corresponding to the event in the perceptual dimension.
Li discloses “wherein the perceptual quality scores represent media qualities perceived by a user while using the plurality of applications in the XR device”. For example, in paragraph [0098], Li teaches video and audio quality becomes poor; Li further teaches video and audio are two different streams from video application and audio application; Li further more teaches when network transmission quality is relatively poor, the VR multimedia may be frozen or video and audio quality becomes poor. In paragraph [0100], Li teaches the video image and audio of the VR multimedia are two different streams from two applications; Li further teaches the interaction experience may also reflect a natural degree of the interaction between the user and the VR multimedia. In paragraph [0109], Li teaches the media quality change event is a change of video quality and audio quality of the VR multimedia due to network transmission quality.
Li discloses “based on the perceptual quality scores, determining, by the XR device, at least one candidate application among the plurality of applications with a different media quality from remaining applications among the plurality of applications”. For example, in paragraph [0098], Li teaches the VR multimedia may be frozen or video and audio quality becomes poor; Li further teaches user experience of the virtual environment of the VR multimedia may be interrupted or experience becomes poor. In paragraph [0100], Li teaches a higher latency indicates more unnatural interaction between the user and the VR multimedia. In paragraph [0109], Li teaches the media quality change event is a change of video quality and audio quality of the VR multimedia due to network transmission quality. In paragraph [0114], Li teaches the VR multimedia player is an application; Li further teaches the VR multimedia player application is embedded on a page of a browser; Li further more teaches the terminal 101 sample, based on different sampling periods, a plurality of parameters in the process of playing the VR multimedia; In addition, Li teaches the VR multimedia, pages, and a browser are multiple applications. In paragraph [0269], Li teaches for each of the at least two perceptual dimensions, determine the process of playing the VR multimedia, i.e. candidate application. In paragraph [0270], Li teaches determining the process of playing the VR multimedia.
Li discloses “determining, by the XR device, a target perceptual quality score for at least one media stream of the at least one candidate application based on the perceptual quality scores, a capability of the XR device and a plurality of network parameters”. For example, in paragraph [0004], Li teaches a mean opinion score, i.e., MOS; determine the experience quality of the VR multimedia based on the MOS that is obtained through calculation. In paragraph [0101], Li teaches if the change of the VR multimedia is inconsistent with the changes of the body posture and the action of the user, relatively poor interaction experience is caused. In paragraph [0109], Li teaches the media quality change event is a change of video quality due to network transmission quality. In paragraph [0127], Li teaches determining experience quality of VR multimedia using the processor 301 and one or more software modules in the program code 308 in the memory 303. In paragraph [0129], Li teaches determining the MOS, i.e. a mean opinion score, of the VR multimedia based on at least two experience scores corresponding to the at least two perceptual dimensions. In paragraph [0130], Li teaches determining experience quality of VR multimedia. In paragraph [0135], Li teaches a network media event affects the enjoyment experience score. In paragraph [0148], Li teaches a MOS of the VR multimedia is determined using an event as a unit; the events include the network media event.
Ren discloses “harmonizing, by the XR device, at least one media parameter of the at least one media stream of the at least one candidate application based on the target perceptual quality score”. For example, in col. 4, lines 15-30, Ren teaches augmented reality, virtual reality, and mixed reality. In col. 8, lines 15-30, Ren teaches one or more composition quality scores are all above or equal to a threshold value. In col. 8, lines 50-60, Ren teaches reducing latency in generating harmonized stitched images. In col. 9, lines 25-35, Ren teaches a color harmonizer 130 harmonizes color across portions of the stitched image data 125 that come from different frames of processed image data 110 to generate harmonized image data 150, e.g., a harmonized 360° surround view visualization, a harmonized panorama. In col. 10, lines 1-15, Ren teaches generating harmonized frames of image data; Ren further teaches color harmonization may be performed at any stage of an image processing pipeline, depending on the implementation. In col. 13, lines 45-55, Ren teaches entire reference and target frames; Ren further teaches applying color harmonization using ground projections of reference and target frames to compute color statistics. In Fig. 5, and col. 14, lines 1-10, Ren teaches FIG. 5 represents color harmonization of the unharmonized stitched image 510 using color statistics computed over ground projections of the reference and target frames. In col. 22, lines 20-30, Ren teaches generating a video stream. In col. 23, lines 60-65, Ren teaches perception, augmented reality, virtual reality, and mixed reality. In col. 34, lines 48-55: Ren teaches multichannel audio over multiple interfaces. In col. 35, lines 45-55, Ren teaches receiving video and input from cameras.
Claims 2-6, 11-16, and 21 are not allowable due to the similar reasons as discussed above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-6, 11-16, and 21 are rejected under 35 U.S.C. 103 as being unpatentable over Li (US 20210350518 A1) in view of Ren (US 12327413 B2), and further in view of Koblin (US 20200118343 A1).
Regarding to claim 1 (Currently Amended), Li discloses a method performed by an extended reality (XR) device for harmonizing perceptual quality of multiple applications in an XR environment ([0002]: virtual reality, VR, is an extended reality, XR; Fig. 1; [0113]: determine experience quality of VR multimedia; obtain, based on the received VR multimedia obtaining request, VR multimedia; [0114]: an application embedded on a page of a browser; determine experience quality of the VR multimedia; [0115]: a head-mounted display device and a mobile terminal; [0127]: determine experience quality of VR multimedia; [0268]: determine the fidelity experience score, the enjoyment experience score, and the interaction score; [0269]: calculate and output a fidelity experience score, an enjoyment experience score, and an interaction score of the VR multimedia corresponding to the event in the perceptual dimension), the method comprising:
receiving, by the XR device, media streams received respectively from a plurality of applications running in the XR device ([0098]: video and audio are not the same; they are distinct, separate, and complementary media formats; video and audio quality becomes poor; video and audio are two different streams from video application and audio application; when network transmission quality is relatively poor, the VR multimedia may be frozen or video and audio quality becomes poor; [0100]: the video image and audio of the VR multimedia are two different streams from two applications; [0109]: video quality and audio quality of the VR multimedia; Fig. 1; [0113]: the server 102 obtains, based on the received VR multimedia obtaining request, VR multimedia, i.e. media stream, and related information required for playing the VR multimedia, and send the VR multimedia and the related information to the terminal 101; [0117]: the terminal 101 receives the VR multimedia, i.e. media stream, and the interactive action of the user);
determining, by the XR device, perceptual quality scores respectively for the media streams received from the plurality of applications ([0098]: video and audio quality becomes poor; video and audio are two different streams from video application and audio application; when network transmission quality is relatively poor, the VR multimedia may be frozen or video and audio quality becomes poor; [0100]: the video image and audio of the VR multimedia are two different streams from two applications; Fig. 1; [0113]: determine experience quality of VR multimedia; [0127]: determine experience quality of VR multimedia; [0129]: calculate an experience score corresponding to each of at least two perceptual dimensions; [0134]: determine a fidelity experience score, an enjoyment experience score, and an interaction experience score of the VR multimedia; [0268]: determine the fidelity experience score, the enjoyment experience score, and the interaction score; [0269]: calculate and output a fidelity experience score, an enjoyment experience score, and an interaction score of the VR multimedia corresponding to the event in the perceptual dimension), wherein the perceptual quality scores represent media qualities perceived by a user while using the plurality of applications in the XR device ([0098]: video and audio are not the same; they are distinct, separate, and complementary media formats; video and audio quality becomes poor; video and audio are two different streams from video application and audio application; when network transmission quality is relatively poor, the VR multimedia may be frozen or video and audio quality becomes poor; [0100]: the video image and audio of the VR multimedia are two different streams from two applications; the interaction experience may also reflect a natural degree of the interaction between the user and the VR multimedia; [0109]: the media quality change event is a change of video quality and audio quality of the VR multimedia due to network transmission quality);
based on the perceptual quality scores, determining, by the XR device, at least one candidate application among the plurality of applications with a different media quality from remaining applications among the plurality of applications ([0098]: the VR multimedia may be frozen or video and audio quality becomes poor; user experience of the virtual environment of the VR multimedia may be interrupted or experience becomes poor; [0100]: a higher latency indicates more unnatural interaction between the user and the VR multimedia; [0109]: the media quality change event is a change of video quality and audio quality of the VR multimedia due to network transmission quality; [0114]: the VR multimedia player is an application; the VR multimedia player application is embedded on a page of a browser; the terminal 101 sample, based on different sampling periods, a plurality of parameters in the process of playing the VR multimedia; the VR multimedia, pages, and a browser are multiple applications; [0269]: for each of the at least two perceptual dimensions, determine the process of playing the VR multimedia, i.e. candidate application; [0270]: determine the process of playing the VR multimedia);
determining, by the XR device, a target perceptual quality score for at least one media stream of the at least one candidate application based on the perceptual quality scores, a capability of the XR device and a plurality of network parameters ([0004]: a mean opinion score, i.e., MOS; determine the experience quality of the VR multimedia based on the MOS that is obtained through calculation; [0101]: if the change of the VR multimedia is inconsistent with the changes of the body posture and the action of the user, relatively poor interaction experience is caused; [0109]: the media quality change event is a change of video quality due to network transmission quality; [0127]: determine experience quality of VR multimedia using the processor 301 and one or more software modules in the program code 308 in the memory 303; [0129]: determine the MOS, i.e. a mean opinion score, of the VR multimedia based on at least two experience scores corresponding to the at least two perceptual dimensions; [0130]: determine experience quality of VR multimedia; [0135]: a network media event affects the enjoyment experience score; [0148]: a MOS of the VR multimedia is determined using an event as a unit; the events include the network media event; [0178]; [0182]); and
Li fails to explicitly disclose:
each of the plurality of applications providing different services;
harmonizing, by the XR device, at least one media parameter of the at least one media stream of the at least one candidate application based on the target perceptual quality score.
In same field of endeavor, Ren teaches:
harmonizing, by the XR device, at least one media parameter of the at least one media stream of the at least one candidate application based on the target perceptual quality score (col. 4, lines 15-30: augmented reality, virtual reality, and mixed reality; col. 8, lines 15-30: one or more composition quality scores are all above or equal to a threshold value; col. 9, lines 25-35: a color harmonizer 130 harmonizes color across portions of the stitched image data 125 that come from different frames of processed image data 110 to generate harmonized image data 150, e.g., a harmonized 360° surround view visualization, a harmonized panorama; col. 10, lines 1-15: generate harmonized frames of image data; color harmonization may be performed at any stage of an image processing pipeline, depending on the implementation; col. 13, lines 45-55: entire reference and target frames; apply color harmonization using ground projections of reference and target frames to compute color statistics; Fig. 5; col. 14, lines 1-10: FIG. 5 represents color harmonization of the unharmonized stitched image 510 using color statistics computed over ground projections of the reference and target frames; col. 22, lines 20-30: generate a video stream; col. 23, lines 60-65: perception, augmented reality, virtual reality, and mixed reality).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Li to include harmonizing, by the XR device, at least one media parameter of the at least one media stream of the at least one candidate application based on the target perceptual quality score as taught by Ren. The motivation for doing so would have been to improve visual quality of stitched images, to reduce latency in generating harmonized stitched images; to perform any known computer vision loop closure technique and global optimization to improve color harmonization; to improve the results of color harmonization over applying it across different frames in different or nonlinear light spaces as taught by Ren in col. 8, lines 15-30, col. 8, lines 50-60, col. 10, lines 45-55 and col. 19, lines 35-45.
Li in view of Ren fails to explicitly disclose: each of the plurality of applications providing different services.
In same field of endeavor, Koblin teaches:
each of the plurality of applications providing different services (Fig. 2A; [0209]: the different mechanisms are used for different types of AR experiences; each of the different mechanisms is an application and provides different services;
PNG
media_image1.png
286
234
media_image1.png
Greyscale
;
PNG
media_image2.png
144
220
media_image2.png
Greyscale
; Fig. 2A; [0210]: the AR App 232 orchestrates the use of various mechanisms; Fig. 2A; [0211]: animation and manipulation use various of the AR app's mechanisms; Fig. 2B; [0215]: an AR system includes a backend platform 256 that may provide resources or mechanisms to support the AR app 232 on one or more devices; [0218]: a backend platform 256 includes services; speech and/or voice recognition mechanisms 276, speech and/or voice augmentation mechanism(s) 278, and language translation mechanism(s) 280 provide different services; [0220]: obtain language translation services from another platform; Fig 2A; [0229]: the communication mechanism 240, the augmenter mechanism 234, the facial recognition mechanism 236, the facial manipulation mechanism 238, the voice manipulation mechanism 246 and the animation mechanism 242 are different applications; communication application provides communication services; the facial recognition application provides facial recognition service; [0348]: the software running on one or more computers implements the applications, services, mechanisms, operations, and acts; [0362]: Execution of applications 1722 produces processing functionality of the service related to the applications; [0238]: each user's display shows a real time view from their rear camera, augmented with an image based on a real time view of the other user's front camera).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Li and Ren to include each of the plurality of applications providing different services as taught by Koblin. The motivation for doing so would have been to perform, create, maintain and generally operate a wide variety of different types of AR experiences, interactions, collaborations and environments; to provide services, such as, speech and/or voice recognition, speech and voice augmentation, and language translation; to produce processing functionality of the service related to the applications as taught by Koblin in paragraphs [0210-0211], [0218], and [0362].
Regarding to claim 2 (Original), Li in view of Ren and Koblin discloses the method of claim 1, wherein the at least one media parameter comprises at least one of a bit-rate, a codec, a resolution, a brightness, a frame-complexity, a sample rate, a channel count, a pitch, or a frame-rate (Li; [0004]: a mean opinion score (MOS) video (MOSV) is obtained through calculation based on parameters such as content complexity, a bit rate per pixel, a coding type, a resolution, a video pause rate, and a packet loss rate of the VR multimedia).
Regarding to claim 3 (Previously Presented), Li in view of Ren and Koblin discloses the method of claim 1, wherein the determining of the target perceptual quality score for the at least one media stream of the at least one candidate application comprises:
determining, by the XR device, a bandwidth ratio based on the plurality of network parameters (Li; [0102]: network transmission quality; [0109]: the freeze event indicates that play of the VR multimedia is paused because network transmission quality is relatively poor; [0145]: a relatively poor network condition);
assigning, by the XR device, a weight factor to the at least one candidate application based on the bandwidth ratio (Li; [0004]: weighting operations are performed on the MOSV and the MOSA based on different weights; [0109]: the freeze event indicates that play of the VR multimedia is paused because network transmission quality is relatively poor; [0145]: a relatively poor network condition); and
determining, by the XR device, the target perceptual quality score for the at least one media stream of the at least one candidate application based on the perceptual quality scores, the plurality of network parameters, the assigned weight factor to the at least one candidate application, and the bandwidth ratio (Li; [0004]: a mean opinion score, i.e., MOS; determine the experience quality of the VR multimedia based on the MOS that is obtained through calculation; [0101]: if the change of the VR multimedia is inconsistent with the changes of the body posture and the action of the user, relatively poor interaction experience is caused; [0109]: the media quality change event is a change of video quality and audio quality of the VR multimedia due to network transmission quality; [0127]: determine experience quality of VR multimedia using the processor 301 and one or more software modules in the program code 308 in the memory 303; [0129]: determine the MOS, i.e. a mean opinion score, of the VR multimedia based on at least two experience scores corresponding to the at least two perceptual dimensions; [0130]: determine experience quality of VR multimedia; [0135]: a network media event affects the enjoyment experience score; [0148]: a MOS of the VR multimedia is determined using an event as a unit; the events include the network media event; [0178]; [0182]).
Regarding to claim 4 (Previously Presented), Li in view of Ren and Koblin discloses the method of claim 1, wherein the plurality of network parameters comprises at least one of a network bandwidth, network congestion, a network quality of service (QoS) identifier, capability of at least one server from which the media streams are received, or latency requirements (Li; [0098]: when network transmission quality is relatively poor, the VR multimedia may be frozen or video and audio quality becomes poor; [0100]: network transmission quality; [0109]: the play of the VR multimedia is paused because network transmission quality is relatively poor; video quality and audio quality of the VR multimedia; [0145]: a relatively poor network condition).
Regarding to claim 5 (Previously Presented), Li in view of Ren and Koblin discloses the method of claim 1, wherein the determining of the perceptual quality scores respectively for the media streams received from the plurality of applications comprises:
determining, by the XR device, at least one media parameter of the media streams respectively received from the plurality of applications, and a plurality of XR parameters (Li; [0100]: the video image and audio of the VR multimedia are two different streams from two applications; [0109]: video quality and audio quality of the VR multimedia; [0114]: the terminal 101 sample, based on different sampling periods, a plurality of parameters in the process of playing the VR multimedia; [0131]: obtain a first sensory parameter, a second sensory parameter, and a third sensory parameter of the VR multimedia; [0132-0133]); and
determining, by the XR device, the perceptual quality scores respectively for the media streams received from the plurality of applications based on the at least one media parameter and the plurality of XR parameters (Li; [0100]: the video image and audio of the VR multimedia are two different streams from two applications; [0109]: video quality and audio quality of the VR multimedia; Fig. 1; [0113]: determine experience quality of VR multimedia; [0127]: determine experience quality of VR multimedia; [0129]: calculate an experience score corresponding to each of at least two perceptual dimensions; [0134]: determine a fidelity experience score, an enjoyment experience score, and an interaction experience score of the VR multimedia; [0268]: determine the fidelity experience score, the enjoyment experience score, and the interaction score; [0269]: calculate and output a fidelity experience score, an enjoyment experience score, and an interaction score of the VR multimedia corresponding to the event in the perceptual dimension).
Regarding to claim 6 (Previously Presented), Li in view of Ren and Koblin discloses the method of claim 5, wherein the plurality of XR parameters comprises:
an XR rendering space, and user preferences in the XR rendering space (Ren; Fig. 12A; col. 9, lined 40-55: surround camera; col. 12, lines 40-50: project the reference and target frames 302 onto corresponding portions of a representation of the ground; col. 12, lines 55-67: identify the portion of each projection that lands in an overlapping region between the two projections as the corresponding ground projection for each frame), and
wherein the user preferences in the XR rendering space comprises:
a position of the user in the XR rendering space (Ren; Fig. 12A; Fig. 12B; col. 9, lined 40-55: wide-view cameras 1270 and surround camera;
PNG
media_image3.png
490
790
media_image3.png
Greyscale
);
user's observation axis in the position of the user in the XR rendering space (Ren; Fig. 12A; Fig. 12B; col. 9, lined 40-55: wide-view cameras 1270 and surround camera; col. 27, lines 1-10: cameras with a field of view that include portions of the environment in front of the vehicle; identify forward facing paths and obstacles);
user's observation range in the position of the user in the XR rendering space (Ren; Fig. 12A; Fig. 12B; col. 9, lined 40-55: wide-view cameras 1270 and surround camera;
PNG
media_image3.png
490
790
media_image3.png
Greyscale
); and
user's field of view or focus in the position of the user in the XR rendering space (Ren; Fig. 12A; Fig. 12B; col. 9, lined 40-55: wide-view cameras 1270 and surround camera;
PNG
media_image3.png
490
790
media_image3.png
Greyscale
; col. 27, lines 1-10: cameras with a field of view that include portions of the environment in front of the vehicle; identify forward facing paths and obstacles).
Same motivation of claim 1 is applied here.
Regarding to claim 11 (Currently Amended), Li discloses an extended reality (XR) device of harmonizing perceptual quality of multiple applications in an XR environment, the XR device ([0002]: virtual reality, VR, is an extended reality, XR; Fig. 1; [0113]: determine experience quality of VR multimedia; obtain, based on the received VR multimedia obtaining request, VR multimedia; [0114]: an application embedded on a page of a browser; determine experience quality of the VR multimedia; [0115]: a head-mounted display device and a mobile terminal; [0127]: determine experience quality of VR multimedia; [0268]: determine the fidelity experience score, the enjoyment experience score, and the interaction score; [0269]: calculate and output a fidelity experience score, an enjoyment experience score, and an interaction score of the VR multimedia corresponding to the event in the perceptual dimension) comprising:
one or more processors (Fig. 3; [0118]: at least one processor; [0124]); and
memory storing one or more computer programs including computer-executable instructions that, when executed by the one or more processors, cause the XR device to (Fig. 3; [0121]: the memory 303 may be a read-only memory; memory stores expected program code in a form of instructions or data structures and capable of being accessed by a computer):
In same field of endeavor, Ren teaches a content harmonizer (Fig. 1; col. 9, lines 20-30: the color harmonizer; col. 10, lines 1-15: the color harmonizer);
The rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 11.
Regarding to claim 12 (Original), Li in view of Ren and Koblin discloses the XR device of claim 11,
The rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to reject claim 2 is also used to reject claim 12.
Regarding to claim 13 (Previously Presented), Li in view of Ren and Koblin discloses the XR device of claim 11,
The rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 13.
Regarding to claim 14 (Previously Presented), Li in view of Ren and Koblin discloses the XR device of claim 11,
The rest claim limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject claim 14.
Regarding to claim 15 (Previously Presented), Li in view of Ren and Koblin discloses the XR device of claim 11,
wherein, to determine the perceptual quality score for the at least one media stream received from each application of the plurality of applications, the one or more computer programs further comprise computer-executable instructions that, when executed by the one or more processors, cause the XR device to (same as rejected in claim 11):
The rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 15.
Regarding to claim 16 (Previously Presented), Li in view of Ren and Koblin discloses the XR device of claim 15,
The rest claim limitations are similar to claim limitations recited in claim 6. Therefore, same rational used to reject claim 6 is also used to reject claim 16.
Regarding to claim 21 (Previously Presented), Li in view of Ren and Koblin discloses the method of claim 1,
wherein the XR device harmonizes the at least one media parameter based on whether a capability of the XR device meets a capability threshold, whether the XR device has service negotiation capability, and whether the XR device has a latency requirement (Ren; col. 4, lines 15-30: augmented reality, virtual reality, and mixed reality; col. 8, lines 15-30: one or more composition quality scores are all above or equal to a threshold value; col. 8, lines 35-45: reduce computational demands and therefore reduce latency; col. 8, lines 50-60: reduce latency in generating harmonized stitched images; col. 9, lines 25-35: a color harmonizer 130 harmonizes color across portions of the stitched image data 125 that come from different frames of processed image data 110 to generate harmonized image data 150, e.g., a harmonized 360° surround view visualization, a harmonized panorama; col. 10, lines 1-15: generate harmonized frames of image data; color harmonization may be performed at any stage of an image processing pipeline, depending on the implementation; col. 13, lines 45-55: entire reference and target frames; apply color harmonization using ground projections of reference and target frames to compute color statistics; Fig. 5; col. 14, lines 1-10: FIG. 5 represents color harmonization of the unharmonized stitched image 510 using color statistics computed over ground projections of the reference and target frames; col. 19, lines 25-35: generate frames of harmonized image data 935 using any of the techniques described here; col. 22, lines 20-30: generate a video stream; col. 23, lines 60-65: perception, augmented reality, virtual reality, and mixed reality; col. 29, lines 25-30: power management capabilities; col. 32, lines 60-67: provide a high-bandwidth, low latency SRAM; col. 33, lines 30-40: the PVA's capabilities are a good match for algorithmic domains needing predictable processing, at low power and low latency; need predictable processing, at low power and low latency; col. 33, lines 45-55: the PVA may perform computer stereo vision function on inputs from two monocular cameras).
Same motivation of claim 1 is applied here.
Allowable Subject Matter
Claims 7-10 and 17-20 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Hai Tao Sun whose telephone number is (571)272-5630. The examiner can normally be reached 9:00AM-6:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at 5712727642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/HAI TAO SUN/Primary Examiner, Art Unit 2616
Prosecution Insights