DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claim(s) 1-6, 9, 11-13, 15, 25-27 and 29 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Lakshmikantha et al. (US Patent No. 10446119 “Lakshmikantha”).
Regarding claim 1 Lakshmikantha teaches A method, comprising: receiving, by a wearable device having a display (Fig. 4 element 115-a), a first frame and a second frame, (Col 7 lines 34-37 “After sorting the graphic layers, the processing device may divide the graphic layers into sets of consecutive graphic layers that are of the same type. Types of graphic layers include head-locked graphic layers and environment-locked graphic layers”),
The first frame representing a first object fixed to location in world coordinates, the second frame representing a second object fixed to a location in an area of the display (Abstract “Methods, systems, and devices for split rending of multiple graphic layers are described. An extended reality (XR) system may include a processing device that generates and renders multiple graphic layers and a display device that displays the graphic layers to create a virtual environment”. Col 7 lines 38-46 “ A head-locked graphic layer may be a graphic layer that remains at the same place in the user's view (e.g., in the same place on the display) regardless of where the user is looking. For example, head-locked graphic layer 210 may include head-locked objects (e.g., game information and statistics) that remain in the lower-right portion of the user's view, regardless of the direction of the user's gaze. In contrast, an environment-locked graphic layer may be a graphic layer that changes based on the user's head position. Graphic layers 205 may be examples of environment-locked layers”);
receiving data from a sensor of the wearable device, the data representing a change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device; (Col 10 lines 23-33 “At 405, display device 115-a may send interaction information to processing device 300-a. The interaction information may include information about the position and movements of the user 401. In some cases, the interaction information includes an indication of the user's head position, movement, or gaze direction.” As world locked frames ( first frame) and head-locked frame (second frame) are being sent at different frame rates, and the sensor of display device is collecting position and movement of user head, step 405 receives data from a sensor of the wearable device, the data representing a change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device.).
repositioning the first frame in the world coordinates based on the change in position to produce a warped first frame (Col 13 lines 9-14 “At 455, after receiving M composite layers in M streams, display device 115-a may perform asynchronous time warp (ATW) on one or more of the composite layers. For example, display device 115-a may perform ATW on the environment-locked graphic layers included in one of the composite layers” See Fig. 4, step 455 has two inputs , one is M streams which has environment locked layer and head-locked layers and another input comes from step 405 which includes position change information of head of the user. Based on these two inputs step 455 is applying ATW on world locked layers or repositioning the world locked layer.);
combining the warped first frame and the second frame to produce a combined image (Col 7 lines 51-58 “ head-locked layers and environment-locked layers may be rendered onto separate graphics layers (e.g., a head-locked graphic layer, which may also be referred to as an overlay layer, may be composited on top of the rendered view constructed from the environment-locked layers)”); and
displaying the combined image in the display (Col 13 lines 34-38 “ At 460, display device 115-a may display the M composite layers by displaying the N graphic layers represented by the M composite layers. Thus, display device 115-a may display to the user 401 a rendered image or video generated from the N graphic layers”).
Regarding claim 11 Lakshmikantha teaches A method, comprising:
providing, by a companion device (Fig. 4 element 300-a), a first frame and a second frame (Col 10 lines 29-36 “At 410, processing device 300-a may generate N graphic layers, where N is greater than one. One or more of the graphic layers may be generated based on, or in response to, the interaction information received from display device 115-a. For example, the processing device 300-a may generate one or more environment-locked graphic layers responsive to information about the user's head position. The processing device 300-a may also generate one or more head-locked layers”);
encoding the first frame to produce a first encoded image; encoding the second frame to produce aa second encoded image (Col 12 lines 58-62 “ At 445, processing device 300-a may encode the M composite layers according to the transmission parameters selected at 440. For example, processing device 300-a may encode each composite layer so that it is it can be sent at the common frame rate”. In Fig. 4,three layers of data have been encoded where first two layers have world-locked data and third layer has head-locked data);
transmitting the first encoded image and the second encoded image to a wearable device (Col 13 lines 1-2 “At 450, processing device 300-a may transmit M streams containing M composite layers to display device 115-a”),
the wearable device being configured to reposition the first frame in world coordinate based on a change in position of the wearable device during a time it took to send the first encoded image and the second encoded image to the wearable device (Col 13 lines 9-13 “ At 455, after receiving M composite layers in M streams, display device 115-a may perform asynchronous time warp (ATW) on one or more of the composite layers. For example, display device 115-a may perform ATW on the environment-locked graphic layers included in one of the composite layers”) Col 13 lines 1-2 “At 450, processing device 300-a may transmit M streams containing M composite layers to display device 115-a” See Fig. 4, step 455 has two inputs , one is M streams which has environment locked layer and head-locked layers and another input comes step 405 which includes position change information of head of the user. Based on these two inputs step 455 is applying ATW on world locked layers or repositioning the world locked layer. As world locked frames ( first frame) and head-locked frame are being sent at different frame rates, and the sensor of display device is collecting position and movement of user head, step 405 receives data from a sensor of the wearable device, the data representing a change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device.).
Regarding claim 25 Lakshmikantha teaches A companion device, comprising: memory; and processing circuitry coupled to the memory (Col 5 lines 6-10 “The processing device 110 may include memory, a processor, an output, and a communication module. The memory may be, for example, a random-access memory (RAM), a memory buffer, a hard drive, a database, an erasable programmable read only memory (EPROM), an electrically erasable programmable read only memory (EEPROM”), the processing circuitry being configured to:
provide a first frame and a second frame (Col 10 lines 29-36 “At 410, processing device 300-a may generate N graphic layers, where N is greater than one. One or more of the graphic layers may be generated based on, or in response to, the interaction information received from display device 115-a. For example, the processing device 300-a may generate one or more environment-locked graphic layers responsive to information about the user's head position. The processing device 300-a may also generate one or more head-locked layers”);
encode the first frame to produce a first encoded image; encode the second frame to produce a second encoded image (Col 12 lines 58-62 “ At 445, processing device 300-a may encode the M composite layers according to the transmission parameters selected at 440. For example, processing device 300-a may encode each composite layer so that it is it can be sent at the common frame rate”. In Fig. 4,three layers of data have been encoded where first two layers have world-locked data and third layer has head-locked data);
transmit the first encoded image and the second encoded image to a wearable device (Col 13 lines 1-2 “At 450, processing device 300-a may transmit M streams containing M composite layers to display device 115-a”),
the wearable device being configured to reposition the first frame in world coordinates based on a change in position of the wearable device during a time it took to send the first encoded image and the second encoded image to the wearable device (Col 13 lines 9-13 “ At 455, after receiving M composite layers in M streams, display device 115-a may perform asynchronous time warp (ATW) on one or more of the composite layers. For example, display device 115-a may perform ATW on the environment-locked graphic layers included in one of the composite layers” Col 10 lines 23-33 “At 405, display device 115-a may send interaction information to processing device 300-a. The interaction information may include information about the position and movements of the user 401. In some cases, the interaction information includes an indication of the user's head position, movement, or gaze direction.” As world locked frames ( first frame) and head-locked frame are being sent at different frame rates, and the sensor of display device is collecting position and movement of user head, step 405 receives data from a sensor of the wearable device, the data representing a change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device. See Fig. 4, step 455 has two inputs, one is M streams which has environment locked layer and head-locked layers and another input comes step 405 which includes position change information of head of the user. Based on these two inputs step 455 is applying ATW on world locked layers or repositioning the world locked layer.);
Regarding claim 2 Lakshmikantha teaches, wherein the repositioning is not performed on the HL frame (Col 13 lines 30-32 “ So display device 115-a may selectively apply ATW based on the type of graphic layer (e.g., by applying ATW to environment-locked graphic layers and refraining from applying ATW to head-locked graphic layers)”).
Regarding claim 3 Lakshmikantha teaches receiving, from a companion device, a first encoded first image representing the first frame and a second encoded image representing the second frame (Col 12 lines 58-65 “ At 445, processing device 300-a may encode the M composite layers according to the transmission parameters selected at 440. For example, processing device 300-a may encode each composite layer so that it is it can be sent at the common frame rate, over the selected bandwidth, with the selected bitrate. Since only M composite layers are encoded, as opposed to N graphic layers, processing device 300-a can conserve processing resources during encoding”);
decoding the first encoded image to produce the first frame; and decoding the second encoded image to produce the second frame (Col 12 lines 54-57 “Additionally or alternatively, the bandwidth and/or bit rate for a composite layer may be based on the encode abilities of processing device 300-a and/or the decode abilities of display device 115-a”).
Regarding claims 4, 12 and 26 Lakshmikantha teaches wherein the first encoded image and the second encoded image are received at different times (in Lakshmikantha as transmission happens at different rates so timing is also different).
Regarding claims 5, 13 and 27 Lakshmikantha teaches wherein the first encoded image and the second encoded image are received at different rates (Fig. 4 shows different frame rates for WL and HL).
Regarding claim 6 Lakshmikantha teaches wherein combining the warped first frame and the second frame includes: combining the warped first frame with a most recent version of the second frame (Col 7 lines 53-57 “head-locked layers and environment-locked layers may be rendered onto separate graphics layers (e.g., a head-locked graphic layer, which may also be referred to as an overlay layer, may be composited on top of the rendered view constructed from the environment-locked layers)”.).
Regarding claims 9, 15 and 29 Lakshmikantha teaches wherein the companion device and the wearable device are connected using a Bluetooth connection (Col 4 lines 63-65 “The processing device 110 may additionally, or alternatively, include or be referred to by those skilled in the art as a user equipment (UE), a user device, a smartphone, a BLUETOOTH® device”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 7-8, 14 and 17-23 and 28 are rejected under 35 U.S.C. 103 as being unpatentable over Lakshmikantha in view of Ryan et al. (US Pat. Pub. No. 20190068983 “Ryan”).
Regarding claims 7, 14 and 28 Even though Lakshmikantha teaches wherein the first encoded image and/or the second encoded image are encoded as shown above but doesn’t expressly mention about a common encoding scheme.
Ryan teaches encode image data using a common encoding scheme (“[0023]……. For example, transmitter 105 can utilize a given data compression scheme (e.g., H.264/MPEG-4 AVC) for compressing the data which is sent to receiver 110”);
Ryan and Lakshmikantha are analogous art as both of them are related to image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Lakshmikantha by having a common encoding scheme as taught by Ryan.
The motivation for the above is to use well known and standard encoding scheme.
Regarding claim 8 Lakshmikantha as modified by Ryan teaches, the common encoding scheme comprise a data compression scheme (“[0023]……. For example, transmitter 105 can utilize a given data compression scheme (e.g., H.264/MPEG-4 AVC) for compressing the data which is sent to receiver 110”);
Regarding claim 17 Lakshmikantha teaches A wearable device (Fig. 4 element 115-a), being configured to:
receive a first frame and a second frame (Col 7 lines 34-37 “ After sorting the graphic layers, the processing device may divide the graphic layers into sets of consecutive graphic layers that are of the same type. Types of graphic layers include head-locked graphic layers and environment-locked graphic layers”),
the first frame representing a first object fixed to a location in world coordinates, the second frame representing a second object fixed to a location in area of a display of the wearable device (Abstract “Methods, systems, and devices for split rending of multiple graphic layers are described. An extended reality (XR) system may include a processing device that generates and renders multiple graphic layers and a display device that displays the graphic layers to create a virtual environment”. Col 7 lines 38-46 “ A head-locked graphic layer may be a graphic layer that remains at the same place in the user's view (e.g., in the same place on the display) regardless of where the user is looking. For example, head-locked graphic layer 210 may include head-locked objects (e.g., game information and statistics) that remain in the lower-right portion of the user's view, regardless of the direction of the user's gaze. In contrast, an environment-locked graphic layer may be a graphic layer that changes based on the user's head position. Graphic layers 205 may be examples of environment-locked layers”);
receive data from a sensor of the wearable device, the data representing a change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device; (Col 10 lines 23-33 “At 405, display device 115-a may send interaction information to processing device 300-a. The interaction information may include information about the position and movements of the user 401. In some cases, the interaction information includes an indication of the user's head position, movement, or gaze direction.” The sensor of display device is capturing images at different time (at different frame rates) and the sensor is collecting position of user’s head and movement which has change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device )’
reposition the first frame in the world coordinates based on the change in position to produce a warped first frame (Col 13 lines 9-14 “At 455, after receiving M composite layers in M streams, display device 115-a may perform asynchronous time warp (ATW) on one or more of the composite layers. For example, display device 115-a may perform ATW on the environment-locked graphic layers included in one of the composite layers”);
combine the warped first frame and the second frame to produce a combined image; (Col 7 lines 51-58 “ head-locked layers and environment-locked layers may be rendered onto separate graphics layers (e.g., a head-locked graphic layer, which may also be referred to as an overlay layer, may be composited on top of the rendered view constructed from the environment-locked layers)”); and
displaying the combined image in the display (Col 13 lines 34-38 “ At 460, display device 115-a may display the M composite layers by displaying the N graphic layers represented by the M composite layers. Thus, display device 115-a may display to the user 401 a rendered image or video generated from the N graphic layers”).
However Lakshmikantha doesn’t expressly mention about A wearable device, comprising: memory; and processing circuitry coupled to the memory;
Ryan teaches A wearable device, comprising: memory; and processing circuitry coupled to the memory (“[0018]….. receiver 110 can be a mobile phone, tablet, computer, server, head-mounted display (HMD), television, another type of display, router, or other types of computing or communication devices. [0022] Similar to transmitter 105, the components implemented within receiver 110 include at least RF transceiver module 145, processor 150, memory 155, and antenna 160”);
Ryan and Lakshmikantha are analogous art as both of them are related to image processing.
Therefore it would have been obvious for an ordinary skilled person in the art before the effective filing date of claimed invention to have modified Lakshmikantha by having a wearable device, comprising: memory; and processing circuitry coupled to the memory as taught by Ryan.
The motivation for the above is to have standard component in a wearable device.
Regarding claim 18 Lakshmikantha modified by Ruan teaches wherein the processing circuitry is further configured to: receive, from a companion device, a first encoded image representing the first frame and a second encoded image representing the second frame; decode the first encoded image to produce the first frame; and decode the second encoded image to produce the second frame (Lakshmikantha Col 12 lines 58-65 “ At 445, processing device 300-a may encode the M composite layers according to the transmission parameters selected at 440. For example, processing device 300-a may encode each composite layer so that it is it can be sent at the common frame rate, over the selected bandwidth, with the selected bitrate. Since only M composite layers are encoded, as opposed to N graphic layers, processing device 300-a can conserve processing resources during encoding”. Col 12 lines 54-57 “Additionally or alternatively, the bandwidth and/or bit rate for a composite layer may be based on the encode abilities of processing device 300-a and/or the decode abilities of display device 115-a”).
Regarding claim 19 Lakshmikantha modified by Ruan teaches wherein the first encoded image and the second encoded image are received at different times.( in Lakshmikantha as transmission happens at different rates so timing is also different).
Regarding claim 20 Lakshmikantha modified by Ruan teaches wherein the first encoded image and the second encoded image are received at different rates. (Lakshmikantha Fig. 4 shows different frame rates for WL and HL).
Regarding claim 21 Lakshmikantha modified by Ruan teaches wherein the processing circuitry configured to combine the warped first frame and the second frame is further configured to: combine the warped first frame with a most recent version of the second frame. (Lakshmikantha Col 7 lines 53-57 “head-locked layers and environment-locked layers may be rendered onto separate graphics layers (e.g., a head-locked graphic layer, which may also be referred to as an overlay layer, may be composited on top of the rendered view constructed from the environment-locked layers)”.).
Regarding claim 22 Lakshmikantha modified by Ruan teaches wherein the first encoded image and the second encoded image are encoded using a common encoding scheme. (Ryan “[0023]……. For example, transmitter 105 can utilize a given data compression scheme (e.g., H.264/MPEG-4 AVC) for compressing the data which is sent to receiver 110”);
Regarding claim 23 Lakshmikantha modified by Ruan teaches wherein the companion device and the wearable device are connected using a Bluetooth connection (Lakshmikantha Col 4 lines 63-65 “The processing device 110 may additionally, or alternatively, include or be referred to by those skilled in the art as a user equipment (UE), a user device, a smartphone, a BLUETOOTH® device”).
Response to Arguments
Applicant’s arguments, see remarks pages 9-10, filed 07/07/2026, with respect to rejection of claim 1 under 35 USC 102(a)(1) have been fully considered and are not persuasive. The rejection has been maintained.
Applicant argues see remarks pages 9-10 “The Applicant has amended claim 1 to recite "receiving data from a sensor of the wearable device, the data representing a change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device; repositioning the first frame in the world coordinates based on the change in position to produce a warped first frame." Support for the amendment may be found in the Specification at, e.g., paragraph [0025].
The cited reference, Lakshmikantha, does not teach or suggest at least the above-cited feature of claim 1 as amended. Rather, Lakshmikantha discloses the following. …………..Lakshmikantha discloses that "display device 115-a may send interaction information to processing device 300-a" (Lakshmikantha, C. 10, 11. 23-26.) and "[Asymchronous time warp] may reduce the delay between a user's head motion and the updating of the display to match the user's motion" (Lakshmikantha, c. 13, 11. 9-22). Put another way, Lakshmikantha discloses applying an asynchronous time warp to catch up to the latest head pose to reduce general motion- to-photon delay…….
Nowhere, however, does Lakshmikantha teach or suggest "repositioning the first frame in the world coordinates based on the change in position [of the wearable device during a time it took to send the first frame and the second frame to the wearable device]," as recited in claim 1 as amended. Lakshmikantha is completely silent with regard to repositioning an environment- locked frame based on a change of position of a wearable device during a time it took to send an environment-locked and head-locked frame to a wearable device. Accordingly, claim 1 as amended is not anticipated by Lakshmikantha.”
Examiner replies, Lakshmikantha Fig.4 step 405 and Col 10 lines 23-33 teaches, receiving data from a sensor of the wearable device, the data representing a change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device. The frames are being sent at different frame rate ( at different frame rates) and the sensor is collecting position of user’s head and movement. See Col 10 lines 23-33 “At 405, display device 115-a may send interaction information to processing device 300-a. The interaction information may include information about the position and movements of the user 401. In some cases, the interaction information includes an indication of the user's head position, movement, or gaze direction. As world locked frames ( first frame) and head-locked frame are being sent at different frame rates, and the sensor of display device is collecting position and movement of user head, step 405 receives data from a sensor of the wearable device, the data representing a change in position of the wearable device during a time it took to send the first frame and the second frame to the wearable device.
Lakshmikantha Fig.4 step 455 and Col 13 lines 9-14 teaches, repositioning the first frame in the world coordinates based on change in position to produce a warped first frame. See Col 13 lines 9-14 “At 455, after receiving M composite layers in M streams, display device 115-a may perform asynchronous time warp (ATW) on one or more of the composite layers. For example, display device 115-a may perform ATW on the environment-locked graphic layers included in one of the composite layers” See Fig. 4, step 455 has two inputs , one is M streams which has environment locked layer and head-locked layers and another input comes step 405 which includes position change information of head of the user. Based on these two inputs step 455 is applying ATW on world locked layers or repositioning the world locked layer.
Therefore applicant’s argument is not persuasive.
In response to applicant’s argument for other independent claims 11, 17 and 25, examiner refers applicant to the reply given above for independent claim1 above as there is no other argument for these claims.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SAPTARSHI MAZUMDER whose telephone number is (571)270-3454. The examiner can normally be reached 8 am-4 pm PST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Said Broome can be reached at (571)272-2931. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/SAPTARSHI MAZUMDER/ Primary Examiner, Art Unit 2612
Prosecution Insights