DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 34, 35, 37, 38, and 40 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Ui et al. (US 2016/0358381 A1).
Regarding claim 34, Ui discloses a computing device, comprising: one or more processors to: (Paragraph 0038, head mounted display (HMD) and PC having their own CPUs) receive frames captured by one or more cameras of a head-mounted display device (Paragraphs 0028-0029, image capturing unit capturing images of an environment in a user’s line of sight), wherein the frames are received, by the computing device over a wireless connection; (Paragraph 0033, the PC receives captured images from the HMD wirelessly) render the frames or frame portions; (Paragraph 0034, the PC generates CG images using the received captured images) compress the rendered frames or frame portions; (Paragraph 0035, compressing the generated CG images) transmit the compressed frames or frame portions over the wireless connection to the head-mounted display device; (Paragraph 0063, the PC transmits the compressed CG images to the HMD) monitor a frame rate or latency of transmission over the wireless connection; (Paragraph 0052, the PC determines an image transmission time of the captured images) and adjust a compression process, a rendering process, or a transmission process with regard to the frames or the frame portions such that the frame rate or the latency of transmission satisfy one or more thresholds (Paragraph 0066, compression ratio and/or rendering resolution of the CG images can be adaptively changed to satisfy a delay time to be in a predetermined range while providing a satisfactory image).
Regarding claim 35, Ui discloses wherein the computing device is configured to: monitor a rate at which the frames or the frame portions are rendered by a rendering application on the computing device; (Paragraph 0036, measuring CG rendering processing time) and in response to detecting that the rendering rate is below a threshold, direct the rendering application to reduce a complexity of one or more rendering processes, wherein reducing the complexity of the one or more rendering processes reduces a resolution of the rendered frames or frame portions and increases a rate at which frames or frame portions are rendered (Paragraph 0066, adaptively changing the rendering resolution based on the CG rendering processing time).
Regarding claim 37, Ui discloses wherein the computing device is configured to: monitor a rate at which frame portions are rendered and/or monitor bandwidth usage on the wireless connection; (Paragraph 0062, a delay time for receiving images for producing the rendered CG images is determined) and in response to detecting that the rendering rate is below a rendering threshold or that the bandwidth usage is above a bandwidth threshold, adjust one or more compression processes to increase a compression ratio at which the rendered frames or frame portions are compressed (Paragraph 0062, increasing a compression ratio based on the delay time).
Regarding claim 38, Ui discloses wherein the monitored latency of transmission comprises: a latency to transmit the captured frames from the head-mounted device to the computing device; (Paragraph 0037, captured image transmission time) a latency to generate the compressed frames or frame portions at the computing device; (Paragraph 0036, CG rendering processing time) and a latency to transmit the compressed frames or frame portions to the head-mounted device (Paragraph 0050, CG image transmission time).
Regarding claim 40, Ui discloses wherein the head-mounted display comprises: a headset (Paragraph 0026, HMD); or glasses.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 21, 22, 24, 26-30, 32, and 33 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ui et al. (US 2016/0358381 A1) in view of Ambrus et al. (US 2017/0115488 A1).
Regarding claim 21, Ui discloses a system, comprising: a base station comprising one or more processors; (Paragraphs 0038, PC having its own CPU) and a device, comprising: one or more processors; (Paragraph 0038, head mounted display (HMD) having its own CPU) one or more cameras configured to capture frames that include views of a user's environment; (Paragraphs 0028-0029, image capturing unit capturing images of an environment in a user’s line of sight) and a display subsystem for displaying a virtual view to the user; (Paragraph 0031, display of MR images) wherein the device is configured to: transmit frames captured by the one or more cameras to the base station over a wireless connection; (Paragraph 0033, the PC receives captured images from the HMD wirelessly) wherein the base station is configured to: render the frames or frame portions; (Paragraph 0034, the PC generates CG images using the received captured images) compress the rendered frames or frame portions; (Paragraph 0035, compressing the generated CG images) and transmit the compressed frames or frame portions to the device over the wireless connection; (Paragraph 0063, the PC transmits the compressed CG images to the HMD) wherein the device is configured to decompress the compressed frames or frame portions received from the base station and provide the rendered frames or frame portions to the display subsystem for display; (Paragraph 0065, the HMD decompresses the compressed CG images to obtain the generated CG images for display in a MR image) and wherein the device or the base station is configured to: monitor a frame rate or latency of transmission of the frames, or frame portions, or the compressed frames, or the compressed frame portions; (Paragraph 0052, the PC determines an image transmission time of the captured images) and adjust a compression process, a rendering process, or a transmission process with regard to the frames or the frame portions such that the frame rate or the latency of transmission satisfy one or more thresholds (Paragraph 0066, compression ratio and/or rendering resolution of the CG images can be adaptively changed to satisfy a delay time to be in a predetermined range while providing a satisfactory image). Ui does not clearly disclose a display subsystem for displaying a 3D virtual view to the user. Ambrus discloses a HMD for mixed reality display using 3D models in a virtual environment that models a physical environment (Paragraph 0079). Ambrus’ technique of using a HMD for mixed reality display using 3D models would have been recognized by one of ordinary skill in the art to be applicable to the HMD for displaying CG images in mixed reality of Ui and the results would have been predictable in the HMD display of 3D CG images and models in a virtual environment of a physical environment in mixed reality. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Regarding claim 22, Ui discloses wherein the base station is configured to: monitor a rate at which the frames or the frame portions are rendered by a rendering application on the base station; (Paragraph 0036, measuring CG rendering processing time) and in response to detecting that the rendering rate is below a threshold, direct the rendering application to reduce a complexity of one or more rendering processes, wherein reducing the complexity of the one or more rendering processes reduces a resolution of the rendered frames or frame portions and increases a rate at which the frames or the frame portions are rendered (Paragraph 0066, adaptively changing the rendering resolution based on the CG rendering processing time).
Regarding claim 24, Ui discloses wherein the base station is configured to: monitor a rate at which frames or frame portions are rendered and/or monitor bandwidth usage on the wireless connection; (Paragraph 0062, a delay time for receiving images for producing the rendered CG images is determined) and in response to detecting that the rendering rate is below a rendering threshold or that the bandwidth usage is above a bandwidth threshold, adjust one or more compression processes on the base station to increase a compression ratio at which the rendered frames or frame portions are compressed (Paragraph 0062, increasing a compression ratio based on the delay time).
Regarding claim 26, Ui discloses wherein the monitored latency of transmission of the frames or frame portions comprises: a latency to transmit the captured frames from the device to the base station; (Paragraph 0037, captured image transmission time) a latency to generate the compressed frames or frame portions at the base station; (Paragraph 0036, CG rendering processing time) and a latency to transmit the compressed frames or frame portions to the device (Paragraph 0050, CG image transmission time).
Regarding claim 27, Ui discloses wherein the rendered frames or frame portions comprise a mixed reality view (Paragraph 0004, mixed reality).
Regarding claim 28, Ui discloses wherein the device comprises: a headset (Paragraph 0026, HMD); or glasses.
Regarding claim 29, Ui discloses, wherein the base station is: a desktop computer (Paragraph 0026, PC); a notebook or laptop computer; a pad or tablet device; a smartphone; or a hand-held computing device.
Regarding claim 30, similar reasoning as discussed in claim 21 is applied.
Regarding claim 32, similar reasoning as discussed in claim 27 is applied.
Regarding claim 33, similar reasoning as discussed in claim 28 is applied.
Claim(s) 23 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ui et al. (US 2016/0358381 A1) in view of Ambrus et al. (US 2017/0115488 A1) and further in view of Turner (US 2011/0126250 A1).
Regarding claim 23, Ui in view of Ambrus discloses all limitations as discussed in claim 21. Ui in view of Ambrus does not clearly disclose wherein the base station is configured to: monitor bandwidth usage on the wireless connection between the device and the base station; and in response to detecting that the bandwidth usage is above a threshold, direct the rendering application to reduce a complexity of one or more rendering processes, wherein reducing the complexity of the one or more rendering processes reduces a resolution of the rendered frames or frame portions.
Turner discloses reducing the resolution of images if excessive bandwidth usage for transmitting the images occurs (Paragraph 0051). Turner’s technique of reducing the resolution of images if excessive bandwidth usage for transmitting the images occurs would have been recognized by one of ordinary skill in the art to be applicable to the transmission of CG images with a resolution that is adaptably changed of Ui in view of Ambrus and the results would have been predictable in the transmission of CG images with a resolution that can be adaptively reduced if excessive bandwidth is used. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 36 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ui et al. (US 2016/0358381 A1) in view of Turner (US 2011/0126250 A1).
Regarding claim 36, Ui discloses all limitations as discussed in claim 21. Ui does not clearly disclose wherein the computing device is configured to: monitor bandwidth usage on the wireless connection between the head-mounted device and the computing device; and in response to detecting that the bandwidth usage is above a threshold, direct the rendering application to reduce a complexity of one or more rendering processes, wherein reducing the complexity of the one or more rendering processes reduces a resolution of the rendered frames or frame portions. Turner discloses reducing the resolution of images if excessive bandwidth usage for transmitting the images occurs (Paragraph 0051). Turner’s technique of reducing the resolution of images if excessive bandwidth usage for transmitting the images occurs would have been recognized by one of ordinary skill in the art to be applicable to the transmission of CG images with a resolution that is adaptably changed of Ui and the results would have been predictable in the transmission of CG images with a resolution that can be adaptively reduced if excessive bandwidth is used. Therefore, the claimed subject matter would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention.
Claim(s) 39 is/are rejected under 35 U.S.C. 103 as being unpatentable over Ui et al. (US 2016/0358381 A1).
Regarding claim 39, Ui discloses all limitations as discussed in claim 38. Ui further discloses wherein said adjusting the compression process, the rendering process, or the transmission process is performed in response to the latency exceeding a threshold (Paragraph 0066, adaptively changing the compression ratio and/or rendering resolution based on image transmission and CG rendering processing times). Ui does not clearly disclose wherein the threshold is 1 millisecond or less; however, it would be an obvious design choice to set the thresholds for the image transmission and CG rendering times to be 1 millisecond or less since they are quantities of times that only determines the conditions to perform the adaptive changing and does not affect the function of the changes to compression or rendering.
Allowable Subject Matter
Claims 25 and 31 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 35, the prior art does not clearly disclose the system of claim 21, wherein the device is configured to: monitor the wireless connection to the base station; and in response to detecting that the wireless connection to the base station has been lost: render, by the device, one or more frames that include views of the user's environment based on the frames captured by the one or more cameras; and provide the rendered one or more frames to the display subsystem for display.
Regarding claim 31, the prior art does not clearly disclose the device of claim 30, wherein the device is configured to: monitor the wireless connection to the base station; and in response to detecting that the wireless connection to the base station has been lost: render, by the device, one or more frames that include views of the user's environment based on the frames captured by the one or more cameras; and provide the rendered one or more frames to the display subsystem for display.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Tsujimoto (US 2014/0160164 A1) discloses an image display apparatus that can disconnect from one access point and connect to another.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHI HOANG whose telephone number is (571)270-3417. The examiner can normally be reached Mon-Fri 8:00-5:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, JASON CHAN can be reached at (571)272-3022. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHI HOANG/Primary Examiner, Art Unit 2619