DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-7, 10-16, 19, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Shchur, US Pub. No. 2023/0020454.
Regarding claim 1, Shchur teaches an operation method of a head-mounted display (HMD) device (fig. 1, MR providing device 100), the operation method comprising: obtaining an original image by capturing a real environment (fig. 1, [0049], real space); detecting at least one object included in the original image (fig. 7, [0191]); obtaining depth information of the detected at least one object using the original image (fig. 7, synthesizer 720, [0196]); identifying a target object among the detected at least one object based on the depth information ([0192], the lower part of the right leg); inpainting a region corresponding to the identified target object in the original image (fig. 7, inpainting module 710, [0193]; and displaying the inpainted image ([0194]).
Regarding claim 2, Shchur teaches obtaining the original image based on at least one of a stereo image obtained through a stereo camera included in the HMD device or a prestored panorama image ([0063]).
Regarding claim 3, Shchur teaches identifying at least one object located within a preset distance range from the HMD device among the detected at least one object based on the depth information (fig. 8, [0031]); and identifying the target object among the identified at least one object ([0204-0205]).
Regarding claim 4, Shchur teaches displaying a virtual object (fig. 1); and identifying the preset distance range based on the displayed virtual object ([0061-0064]).
Regarding claim 5, Shchur teaches wherein the identifying the target object comprises identifying an object classified into a preset class as the target object among the identified at least one object (fig. 8, anchor spot).
Regarding claim 6, Shchur teaches obtaining a user input to select at least one of the detected at least one object as an inpainting target (fig. 7, inpainting module 710), wherein the identifying the target object comprises identifying an object selected as the inpainting target as the target object among the identified at least one close object ([0189-0194).
Regarding claim 7, Shchur teaches wherein the obtaining the inpainted image comprises: obtaining a mask map representing an area corresponding to the identified target object in the original image ([0184]); and obtaining the inpainted image by applying the original image and the mask map to an inpainting model for inpainting the target object ([0184]).
Regarding claim 10, Shchur teaches identifying whether inpainting for the identified target object is required based on motion information of a user wearing the HMD device (fig. 7, [0189-0194]), wherein the obtaining the inpainted image comprises, based on identifying that the inpainting is required, performing inpainting for the identified target object (fig. 7, [0189-0194]).
Regarding claim 11, it is a HMD device of claim 1 and is rejected on the same grounds presented above. Shchur additionally teaches a display (HMD 100); a stereo camera (fig. 2, camera 110, [0063]), a memory storing at least one instruction ([0124]); and at least one processor configured to execute the at least one instruction stored in the memory (fig. 2, processor 140).
Regarding claim 12, it has similar limitations to those of claim 3 and is rejected on the same grounds presented above.
Regarding claim 13, it has similar limitations to those of claim 4 and is rejected on the same grounds presented above.
Regarding claim 14, it has similar limitations to those of claim 5 and is rejected on the same grounds presented above.
Regarding claim 15, it has similar limitations to those of claim 6 and is rejected on the same grounds presented above.
Regarding claim 16, it has similar limitations to those of claim 7 and is rejected on the same grounds presented above.
Regarding claim 19, it has similar limitations to those of claim 10 and is rejected on the same grounds presented above.
Regarding claim 20, it is a non-transitory computer-readable recording medium of claim 1 and is rejected on the same grounds presented above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 8, 9, 17 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Shchur (see above), in view of Liu et al. (hereinafter “Liu”), US Pub. No. 2015/0235632.
Regarding claim 8, Shchur fails to explicitly teach obtaining an outer view image representing a second field of view (FOV) wider than a first FOV of the original image; and tracing the detected at least one object moving in and out of the first FOV based on the original image and the outer view image.
However, in the same field of endeavor, Liu teaches virtual objects located in a HMD to provide an augmented reality view wherein the HMD wearer’s total field of view is classified into two or more regions (see fig. 6 and accompanying text; [0050]; in particular, the objects moving in and out of the primary region 604 would be based on where the user is looking).
Therefore, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Shchur to include the feature of Liu. As such, a person having ordinary skill in the art would appreciate the motivation for doing so would have been to provide a device that prevents virtual images that are not distracting and obtrusive (Liu, [0002]).
Regarding claim 9, Liu teaches displaying a user interface indicating identification information of the detected at least one object located outside the first FOV (fig. 6, objects 610 in primary, secondary, and tertiary regions).
Regarding claim 17, it has similar limitations to those of claim 8 and is rejected on the same grounds presented above.
Regarding claim 18, it has similar limitations to those of claim 9 and is rejected on the same grounds presented above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Joo (US Pub. No. 2018/0174344) teaches capturing real environment with HMD and displaying virtual objects based on real environment capture.
Berliner et al. (US Pub. No. 2023/0139626) teaches a head mounted display including an inpainting algorithm.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KENNETH B LEE JR whose telephone number is (571)270-3147. The examiner can normally be reached Mon - Fri 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, William Boddie can be reached at 571-272-0666. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KENNETH B LEE JR/Primary Examiner, Art Unit 2625