DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Information Disclosure Statement
The information disclosure statement (IDS) was filed on September 05, 2025. The submission is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 9-13, 19 and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Han et al., US 20180366091 A1 (hereinafter referred to as “Han”) in further view of Song et al., US 11442274 B2 (hereinafter referred to as “Song”).
Regarding claim 1, Han discloses a wearable device (Han: FIG. 1, the head mounted display (HMD) device 100 may be worn on the user head. ¶ [0046]), comprising: a camera (Han: The detection unit 120 of the HMD 100 may mainly include a camera. ¶ [0058]. Fig. 2B); a display (Han: Display 110, Fig. 2B.); at least one processor comprising processing circuitry (Han: Processor 130, Fig. 2B); and memory storing instructions (Han: RAM/ROM 131, 132 Fig. 2B), wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to:
receiving an input requesting to display an external object located in an external space around the wearable device on the display while a virtual space is displayed on the display (Han: first displaying context on the display and subsequently changing the display based on the location of an object when movement of the object is detected. Fig. 10, S1010-S1030).
identify metadata corresponding to the external object designated by the input (Han: Mapping and location information of a detected object. ¶ [0016]- [0018]); based on the identification of the metadata, identify, from the metadata, a first area regarding movement of the external object (Han: Mapping and location information of a detected object. ¶ [0016]- [0018]); identify a first image portion corresponding to the first area in a first image obtained through the camera (Han: Movement of the object is detected on a corresponding area of the display. ¶ [0019]); and change a first virtual-space portion, corresponding to the first area within the virtual space and displayed on the display, into the first image portion of the first image, wherein the first image portion of the first image includes a visual object corresponding to the external object (Han: Overlay and display image among images photographed by the camera corresponding to position where movement of the object is detected on a corresponding area of the display.¶[0019]).
Han does not disclose receiving an input requesting to display an external object located in an external space around the wearable device on the display while a virtual space is displayed on the display (emphasis added).
However, in the same field of endeavor, Song discloses receiving an input requesting to display an external object located in an external space around the wearable device on the display while a virtual space is displayed on the display (Song: the processor receives a display request to display a graphic object associated with the external electronic device. Col 29:43-47).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Han with receiving an input requesting to display an external object located in an external space around the wearable device on the display while a virtual space is displayed on the display, as taught by Song, in order to naturally display an external object within a limited scope of view of a virtual reality image, as suggested by the reference. Song, Col. 1:33-39.
Regarding claim 2, the combination of Han and Song disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to: while the first image portion of the first image is displayed on the display, obtain a second image through the camera; in the second image, identify a second image portion corresponding to a second area regarding movement of the external object; and based on identifying, from the second image, the second image portion having a position spaced apart from the first image portion of the first image, change a second virtual-space portion within the virtual space displayed on the display into the second image portion of the second image, and wherein the second image portion of the second image includes a visual object corresponding to the external object in the second image (Han: When movement of the object is detected [among subsequently photographed images], a screen state of the display may be changed to provide front image of the HMD device based on a location of the object at S1030. Specifically, the changing at S1030 may overlay and display an image corresponding to position where movement of the object is detected among the images photographed by the camera on an area corresponding to the display. ¶ [0149]- [0151]).
Regarding claim 3, the combination of Han and Song disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to: in a state in which the first image portion of the first image is displayed on the display based on a position of the first image portion in the first image, cease displaying the first image portion of the first image on the display; and based on a position of the second image portion in the second image, display the second image portion of the second image on the display (Han: The changing display state to overlay and display image corresponding to position where movement of the object is detected among the images photographed by the camera on an area corresponding to the display. ¶ [0149]- [0151]. Fig. 3, items 310 and 320, and Fig. 10).
Regarding claim 9, the combination of Han and Song disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to change the first virtual-space portion displayed on the display into the first image portion of the first image by display-overlapping the first image portion of the first image on the first virtual-space portion (Han: The changing display state to overlay and display image corresponding to position where movement of the object is detected among the images photographed by the camera on an area corresponding to the display. ¶ [0149]- [0151]. Fig. 3, items 310 and 320, and Fig. 10).
Regarding claim 10, the combination of Han and Song disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to change the first virtual-space portion displayed on the display into the first image portion of the first image by ceasing display of the first virtual-space portion and displaying the first image portion of the first image (Han: The changing display state to overlay and display image corresponding to position where movement of the object is detected among the images photographed by the camera on an area corresponding to the display. ¶ [0149]- [0151]. Fig. 3, items 310 and 320, and Fig. 10).
Regarding claim 11, claim 1 is sustainably similar to claim 11. Therefore, claim 11 is rejected for the same reasons as claim 1.
Regarding claim 12, claim 2 is sustainably similar to claim 12. Therefore, claim 12 is rejected for the same reasons as claim 2.
Regarding claim 13, claim 3 is sustainably similar to claim 13. Therefore, claim 13 is rejected for the same reasons as claim 3.
Regarding claim 19, claim 9 is sustainably similar to claim 19. Therefore, claim 19 is rejected for the same reasons as claim 9.
Regarding claim 20, claim 1 is sustainably similar to claim 11. Therefore, claim 20 is rejected for the same reasons as claim 1.
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Han in further view of Song and Matsumura et al., US 20090110291 A1 (hereinafter referred to as “Matsumura”).
Regarding claim 4, the combination of Han and Song disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to: based on the metadata and a third image obtained through the camera including the external object (Han: images photographed by the camera used to detect movement of an external object. ¶ [0019]); identify a third image portion of the third image corresponding to the first area (Han: vehicle 310. Fig. 3);
The combination of Han and Song does not disclose identify a third image portion of the third image corresponding to the first area; based on the metadata and the third image, identify a fourth image portion of the third image corresponding to a third area of another part of the external object; and display the third image portion and the fourth image portion on the display, wherein the third image portion of the third image includes a first visual object corresponding to a part of the external object, and the fourth image portion of the third image includes a second visual object corresponding to another part of the external object.
However, in the same field of endeavor, Matsumura discloses identify a third image portion of the third image corresponding to the first area (Matsumura: hand region 150a of first area 150. ¶ [0012]); based on the metadata and the third image, identify a fourth image portion of the third image corresponding to a third area of another part of the external object (Matsumura: arm region 150b of area 150. ¶ [0012]); and display the third image portion and the fourth image portion on the display, wherein the third image portion of the third image includes a first visual object corresponding to a part of the external object (Matsumura: Display hand region. Fig. 13 [0144], [0147]), and the fourth image portion of the third image includes a second visual object corresponding to another part of the external object (Matsumura: Display arm region, Fig. 13 ¶¶ [0144], [0147])).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Song with identifying a third image portion of the third image corresponding to the first area; based on the metadata and the third image, identifying a fourth image portion of the third image corresponding to a third area of another part of the external object; and displaying the third image portion and the fourth image portion on the display, wherein the third image portion of the third image includes a first visual object corresponding to a part of the external object, and the fourth image portion of the third image includes a second visual object corresponding to another part of the external object, as taught by Matsumura, in order to provide a realistic MR experience as taught by the reference. Matsumura, ¶ [0006].
Regarding claim 14, claim 4 is sustainably similar to claim 14. Therefore, claim 14 is rejected for the same reasons as claim 4.
Claims 5-8 and 15-18 are rejected under 35 U.S.C. 103 as being unpatentable over Han in further view of Song and Matsumura in further view of Li US 20200250832 A1 (hereinafter referred to as “Li”).
Regarding claim 5, the combination of Han, Song and Matsumura does not disclose the wearable device of claim 4, wherein a distance between the third image portion of the third image and the fourth image portion of the third image is greater than or equal to a reference value, and wherein the third image portion of the third image and the fourth image portion of the third image are spaced apart from each other, and wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to cause the wearable device to store, in the memory, information indicating a segmentation of the external object with respect to the metadata for the third image portion and the metadata for the fourth image portion.
However, Li discloses wherein a distance between the third image portion of the third image and the fourth image portion of the third image is greater than or equal to a reference value, and wherein the third image portion of the third image and the fourth image portion of the third image are spaced apart from each other (Li: Method for detecting and tracking multiple moving objects in successive frames. A plurality of split objects (i.e., image portions) are identified from the detected plurality of moving region Abstract. Fig. 4A. Temporal information of different objects engaged in motion in a captured image sequence. Fig. 3B), and wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to cause the wearable device to store, in the memory, information indicating a segmentation of the external object with respect to the metadata for the third image portion and the metadata for the fourth image portion (Li: Temporal information of different objects engaged in motion in a captured image sequence. Fig. 3B).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Matsumura with wherein a distance between the third image portion of the third image and the fourth image portion of the third image is greater than or equal to a reference value, and wherein the third image portion of the third image and the fourth image portion of the third image are spaced apart from each other, and wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to cause the wearable device to store, in the memory, information indicating a segmentation of the external object with respect to the metadata for the third image portion and the metadata for the fourth image portion, as taught by Li, in order to improve object recognition.
Regarding claim 6, the combination of Han, Song and Matsumura do not disclose the wearable device of claim 5, wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to: in response to identifying at least one of (i) a part of the external object corresponding to the third image portion of the third image or (ii) another part of the external object corresponding to the fourth image portion of the third image, identify, based on the information indicating the segmentation of the external object stored in the metadata, the third image portion including the first visual object corresponding to the part of the external object and the fourth image portion including the second visual object corresponding to the other part of the external object (Li: Method for detecting and tracking multiple moving objects in successive frames. A plurality of split objects (i.e., image portions) are identified from the detected plurality of moving region Abstract. Fig. 4A. Temporal information of different objects engaged in motion in a captured image sequence. Fig. 3B).
Regarding claim 7, the combination of Han, Song and Matsumura does not disclose the wearable device of claim 4, wherein the external object is a first external object, and wherein the instructions, when executed by the at least one processor, individually and/or collectively, cause the wearable device to: in a state of displaying the first image portion together with a fifth image portion of the first image corresponding to an area regarding movement of a second external object, identify a position of the fifth image portion and a position of the first image portion in the first image; based on identifying a distance between the position of the first image portion and the position of the fifth image portion within the first image, display a sixth image portion of the first image that includes the first image portion and the fifth image portion, on the display; and store, in the metadata corresponding to the first external object and the second external object, information indicating a combination of the first external object and the second external object in the memory.
However, Li discloses in a state of displaying the first image portion together with a fifth image portion of the first image corresponding to an area regarding movement of a second external object, identify a position of the fifth image portion and a position of the first image portion in the first image; based on identifying a distance between the position of the first image portion and the position of the fifth image portion within the first image, display a sixth image portion of the first image that includes the first image portion and the fifth image portion, on the display; and store, in the metadata corresponding to the first external object and the second external object, information indicating a combination of the first external object and the second external object in the memory (Li: Display and Memory. ¶¶ [0057]- [0059], Figs. 3B, 4A).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Matsumura with in a state of displaying the first image portion together with a fifth image portion of the first image corresponding to an area regarding movement of a second external object, identify a position of the fifth image portion and a position of the first image portion in the first image; based on identifying a distance between the position of the first image portion and the position of the fifth image portion within the first image, display a sixth image portion of the first image that includes the first image portion and the fifth image portion, on the display; and store, in the metadata corresponding to the first external object and the second external object, information indicating a combination of the first external object and the second external object in the memory, as taught by Li, in order to improve object detection in images. Li, ¶ [0003].
Regarding claim 8, Han, Song and Matsumura do not disclose the wearable device of claim 7, wherein: the first image portion of the first image and the fifth image portion of the first image at least partially overlap; and the sixth image portion of the first image includes the first image portion, the fifth image portion, and a portion connecting the first image portion and the fifth image portion.
However, in the same field of endeavor, Li discloses the first image portion of the first image and the fifth image portion of the first image at least partially overlap (Li: At 412, a plurality of merged objects may be identified from the detected plurality of moving regions. ¶ [0097]); and the sixth image portion of the first image includes the first image portion, the fifth image portion, and a portion connecting the first image portion and the fifth image portion (Li: At 412, a plurality of merged objects may be identified from the detected plurality of moving regions. At 414, a plurality of split objects may be identified from the detected plurality of moving regions and the identified plurality of merged objects. ¶¶ [0097]- [0098]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of Matsumura with the first image portion of the first image and the fifth image portion of the first image at least partially overlap; and the sixth image portion of the first image includes the first image portion, the fifth image portion, and a portion connecting the first image portion and the fifth image portion, as taught by Li in order to improve object detection in images. Li, ¶ [0003].
Regarding claim 15, claim 5 is sustainably similar to claim 15. Therefore, claim 15 is rejected for the same reasons as claim 5.
Regarding claim 16, claim 6 is sustainably similar to claim 16. Therefore, claim 16 is rejected for the same reasons as claim 6.
Regarding claim 17, claim 7 is sustainably similar to claim 17. Therefore, claim 17 is rejected for the same reasons as claim 7.
Regarding claim 18, claim 8 is sustainably similar to claim 18. Therefore, claim 18 is rejected for the same reasons as claim 8.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DEIRDRE L BEASLEY whose telephone number is (571)270-0452. The examiner can normally be reached Monday-Friday 8 a.m. -5 p.m.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chris Kelley can be reached at (571) 272-7331. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/DLB/Patent Examiner, Art Unit 2482
/BEHROOZ M SENFI/Primary Examiner, Art Unit 2482
Prosecution Insights