DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . This Office Action is responsive to the communications filed on 2 March 2026. Claims 1-5, 7-14 and 17-19 are pending.
Priority
Acknowledgment is made of applicant’s claim for foreign priority under 35 U.S.C. 119 (a)-(d). The certified copy has been filed in parent Application No. 10-2022-0152098, filed on 14 Nov 2022.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-4, 7-8, 11-13, 17 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 2019/0073832 A1) in view of Cappello et al. (Hereinafter, Cappello, US 2021/0183146 A1), Bailey et al. (Hereinafter, Bailey, US 2021/0256769 A1), and further in view of Wang et al. (Hereinafter, Wang, US 2020/0169586 A1).
Per claim 1, Kim discloses a wearable device (e.g., head-mounted display device (415, 417) as shown in Fig. 4; paragraph [0066], “ The system block diagram (400) for the HoloWalks system in FIG. 4 also incorporates a HoloWalks Viewer module (411), which comprises one or more software sub-modules executed in a CPU, a GPU, and/or a memory unit of a viewer's head-mounted display device (415, 417) or another portable electronic device (e.g. a smart phone, a table computer, etc.) capable of displaying mixed-reality environments via camera-enabled mobile applications... “), comprising:
displays arranged to face eyes of user wearing the wearable device (paragraph [0054]; paragraph [0066], “… In context of various embodiments of the present invention, HoloWalks Events are immersive mixed-reality scenarios that are executed and displayed in the viewer's head-mounted display device (415, 417) or another portable electronic device operatively connected to the HoloWalks Viewer module (411) and/or the rest of the components (i.e. 409, 403, 401) in the HoloWalks system. ”); and
at least one processor including processing circuitry(paragraphs paragraph [0010]; paragraph [0047], “… The newly-created native 3D map file (101) is stored in a 3D mixed-reality map database (113) executed in a memory and a CPU of a 3D map graphics-processing computer server (111). “; paragraph [0062]);
memory storing instructions, that, when executed by the at least one processor individually or collectively(paragraphs paragraph [0010]; paragraph [0047], “… The newly-created native 3D map file (101) is stored in a 3D mixed-reality map database (113) executed in a memory and a CPU of a 3D map graphics-processing computer server (111). “; paragraph [0062]), cause the electronic device to:
identify an input indicating entry into a first portion of a virtual space (e.g., 500B as shown in Fig. 5B; paragraph [0061]; paragraph[0069], “FIG. 5B shows the MR experience designer selecting a desired spot within the three-dimensional (3D) map in a mixed-reality (MR) experience construction interface (500B) to initiate a mixed-reality holographic guide content creation, in accordance with an embodiment of the invention .. “; paragraph [0078]),
identify a plurality of multimedia contents which is displayable through the displays while a location of an avatar representing the user wearing the wearable device is within the first portion of the virtual space, based on entry into the first portion of the virtual space(e.g., 500C as shown in Fig. 5C; paragraph [0069], “… Furthermore, the MR experience designer subsequently places holographic contents and overlays user interaction elements (500C) in the desired spot within the 3D map by utilizing the mixed-reality (MR) experience construction interface in the HoloWalks system, as shown in FIG. 5C ... “), wherein the plurality of multimedia contents includes a first multimedia content arranged in the first portion of the virtual space(e.g., overlays of user interaction elements (‘LIST HOLOPEOPLE’) as shown in Fig. 5C ) and a second multimedia content arranged in a second portion of the virtual space (e.g., mixed-reality environment as shown in 5A-5D; paragraph [0071]); but does not expressly disclose:
the first portion by a barrier structure in the virtual space,
obtain priority order for locating in the first portion of the virtual space outputting the first multimedia content and the second multimedia content based on user profile information of the user corresponding to the wearable device, based on the priority order and the location of the avatar, identify a first position and a second position in the first portion of the virtual space, where the first multimedia content from the first portion of the virtual space and the second multimedia content from the second portion of the virtual space are both arranged in a
display, through the display, a screen the FoV of the user of the first portion of the virtual space, the screen including a first visual object representing the first multimedia content at the first position based on the priority order, based on the priority order instead of displaying second visual object at the second portion of the virtual space, and a third visual object indicating a position where the second multimedia content corresponding to the second visual object are placed, wherein one of the first position and the second position having a higher priority order is located closer to the avatar representing the user.
Cappello discloses:
the first portion by a barrier structure in the virtual space(e.g., geometry 130 as shown in Fig. 1; paragraph [0027], “… In this Figure, the geometry 130 is taken as being walls within the environment 100 although of course this is entirely exemplary—the geometry 130 may be any object or the like. “ ).
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the content generation system and method of Capello with the mixed-reality space map creation device of Kim for generating a more efficient representation of the content as suggested by Cappello (paragraph [0006]).
Bailey discloses:
obtain priority order for locating in the first portion of the virtual space outputting the first multimedia content and the second multimedia content based on user profile information of the user corresponding to the wearable device, based on the priority order and the location of the avatar, identify a first position and a second position in the first portion of the virtual space, where the first multimedia content from the first portion of the virtual space and the second multimedia content from the second portion of the virtual space are both arranged in a
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the session manager of Bailey with the mixed-reality space map creation device of Kim and Cappello for improving realism as suggested by Bailey (paragraph [0008]).
Wang discloses:
display, through the display, a screen the FoV of the user of the first portion of the virtual space, the screen including a first visual object representing the first multimedia content at the first position based on the priority order, based on the priority order instead of displaying second visual object at the second portion of the virtual space, and a third visual object indicating a position where the second multimedia content corresponding to the second visual object are placed, wherein one of the first position and the second position having a higher priority order is located closer to the avatar representing the user(Abstract; paragraph [0003-0006]; paragraph [0016] ).
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the virtual co-experiencing systems of Wang with the mixed-reality space map creation device of Kim, Cappello, Bailey for improving information display in a virtual reality environment.
Per claim 2, Kim, Cappello, Bailey, and Wang disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
identify properties about the plurality of multimedia contents based on embedding video information and audio information included in each of the plurality of multimedia contents(Kim, e.g., steps 202-204 as shown in Fig. 2; paragraph [0050] describes embedded data used to identify properties about the plurality of multimedia contents.), and
obtain the priority order based on the profile information related to the properties(Kim, paragraph [0009]; paragraph [0010]; Steps 206 or Step 208 as shown in Fig. 2; Examiner’s Note: Kim discloses obtaining information related to arranging objects in a 3D mixed-reality environment from a file. ).
Per claim 3, Kim, Cappello, Bailey, and Wang disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device: obtain the priority order for outputting the plurality of multimedia contents, using the profile information of the user indicating a priority for each of the properties (Bailey, Abstract; paragraph [0009];paragraph [0021]).
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the session manager of Bailey with the mixed-reality space map creation device of Kim and Cappello for improving realism as suggested by Bailey (paragraph [0008]).
Per claim 4, Kim, Cappello, Bailey, and Wang disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to:
select at least one multimedia content among the plurality of multimedia content, based on the obtained order(Kim, paragraph [0026]; paragraph [0049]); and
obtain the first and the second visual objects representing the selected at least one multimedia content (Kim, paragraph [0026]; paragraph [0049]).
Per claim 7, Kim, Cappello, Bailey, and Wang disclose the wearable device of claim 1, wherein the instructions, when executed by the
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the virtual co-experiencing systems of Wang with the mixed-reality space map creation device of Kim, Cappello, Bailey for improving information display in a virtual reality environment.
Per claim 8, Kim, Cappello, Bailey, and Wang disclose the wearable device of claim 1, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: update at least one of the first visual object or the second visual object, based on identifying an interaction between the user and the first visual object or the second visual object(Kim, paragraphs [0089-0091).
Per claim 11, Kim discloses a method of a wearable device comprising:
identifying an input indicating entry into a first portion of a virtual space (e.g., 500B as shown in Fig. 5B; paragraph [0061]; paragraph[0069], “FIG. 5B shows the MR experience designer selecting a desired spot within the three-dimensional (3D) map in a mixed-reality (MR) experience construction interface (500B) to initiate a mixed-reality holographic guide content creation, in accordance with an embodiment of the invention .. “; paragraph [0078]),
identifying a plurality of multimedia contents which is displayable through the displays while a location of an avatar representing the user wearing the wearable device is within the first portion of the virtual space, based on entry into the first portion of the virtual space(e.g., 500C as shown in Fig. 5C; paragraph [0069], “… Furthermore, the MR experience designer subsequently places holographic contents and overlays user interaction elements (500C) in the desired spot within the 3D map by utilizing the mixed-reality (MR) experience construction interface in the HoloWalks system, as shown in FIG. 5C ... “), wherein the plurality of multimedia contents includes a first multimedia content arranged in the first portion of the virtual space(e.g., overlays of user interaction elements (‘LIST HOLOPEOPLE’) as shown in Fig. 5C ) and a second multimedia content arranged in a second portion of the virtual space (e.g., mixed-reality environment as shown in 5A-5D; paragraph [0071]); but does not expressly disclose:
the first portion by a barrier structure in the virtual space,
obtaining priority order for locating in the first portion of the virtual space outputting the first multimedia content and the second multimedia content based on user profile information of the user corresponding to the wearable device, based on the priority order and the location of the avatar, identify a first position and a second position in the first portion of the virtual space, where the first multimedia content from the first portion of the virtual space and the second multimedia content from the second portion of the virtual space are both arranged in a
displaying, through the display, a screen the FoV of the user of the first portion of the virtual space, the screen including a first visual object representing the first multimedia content at the first position based on the priority order, based on the priority order instead of displaying second visual object at the second portion of the virtual space, and a third visual object indicating a position where the second multimedia content corresponding to the second visual object are placed, wherein one of the first position and the second position having a higher priority order is located closer to the avatar representing the user.
Cappello discloses:
the first portion by a barrier structure in the virtual space(e.g., geometry 130 as shown in Fig. 1; paragraph [0027], “… In this Figure, the geometry 130 is taken as being walls within the environment 100 although of course this is entirely exemplary—the geometry 130 may be any object or the like. “ ).
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the content generation system and method of Capello with the mixed-reality space map creation device of Kim for generating a more efficient representation of the content as suggested by Cappello (paragraph [0006]).
Bailey discloses:
obtaining priority order for locating in the first portion of the virtual space outputting the first multimedia content and the second multimedia content based on user profile information of the user corresponding to the wearable device, based on the priority order and the location of the avatar, identify a first position and a second position in the first portion of the virtual space, where the first multimedia content from the first portion of the virtual space and the second multimedia content from the second portion of the virtual space are both arranged in a
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the session manager of Bailey with the mixed-reality space map creation device of Kim and Cappello for improving realism as suggested by Bailey (paragraph [0008]).
Wang discloses:
displaying, through the display, a screen the FoV of the user of the first portion of the virtual space, the screen including a first visual object representing the first multimedia content at the first position based on the priority order, based on the priority order instead of displaying second visual object at the second portion of the virtual space, and a third visual object indicating a position where the second multimedia content corresponding to the second visual object are placed, wherein one of the first position and the second position having a higher priority order is located closer to the avatar representing the user(Abstract; paragraph [0003-0006]; paragraph [0016] ).
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the virtual co-experiencing systems of Wang with the mixed-reality space map creation device of Kim, Cappello, Bailey for improving information display in a virtual reality environment.
Per claim 12, Kim, Cappello, Bailey, and Wang disclose the method of the wearable device of claim 11, further comprising: identifying the properties about the plurality of multimedia contents based on embedding video information and audio information included in each of the plurality of multimedia contents(Kim, e.g., steps 202-204 as shown in Fig. 2; paragraph [0050] describes embedded data used to identify properties about the plurality of multimedia contents.).
Per claim 13, Kim, Cappello, Bailey, and Wang disclose the method of the wearable device of claim 11, further comprising: obtaining the priority order using the profile information of the user indicating a priority for each of the properties(Kim, paragraph [0009]; paragraph [0010]; Steps 206 or Step 208 as shown in Fig. 2; Examiner’s Note: Kim discloses obtaining information related to arranging objects in a 3D mixed-reality environment from a file. ).
Per claim 17, Kim, Cappello, Bailey, and Wang disclose the method of the wearable device of claim 11, further comprising: updating at least one of the first visual object or the second visual object, based on identifying an interaction between the user and the first visual object or the second visual object(Kim, paragraphs [0089-0091]).
Per claim 19, Kim, Cappello, Bailey, and Wang disclose the method of the wearable device of claim 11, further comprising: selecting the first multimedia content or the second multimedia content according to the obtained order(Kim, paragraph [0026]; paragraph [0049]);.
Claims 5 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 2019/0073832 A1) in view of Cappello et al. (Hereinafter, Cappello, US 2021/0183146 A1), Bailey et al. (Hereinafter, Bailey, US 20210/256769 A1), Wang et al. (Hereinafter, Wang, US 2020/0169586 A1), and further view of Ziman (US 2021/0011607 A1).
Per claim 5, Kim, Cappello, Bailey, and Wang disclose the wearable device of claim 1, but do not expressly disclose wherein the profile information includes at least one of identification information of the user or information indicating browsing of each of the plurality of multimedia contents.
Ziman discloses wherein the profile information includes at least one of identification information of the user or information indicating browsing of each of the plurality of multimedia contents(Ziman, paragraph [0187];paragraph [0205]; Ziman discloses user interface elements for identifying and modifying information. ).
It would have been obvious for a person of ordinary skill in the art before the effective fling date of the claimed invention to use the interactive virtual world of Ziman with the mixed-reality space map creation device of Kim, Cappello, Bailey, and Wang to improve user interactions in a virtual space.
Per claim 14, Kim, Cappello, Bailey, and Wang disclose the method of the wearable device of claim 11, but do not expressly disclose the method as further comprising: obtaining the order, based on the profile information including at least one of identification information of the user or information indicating browsing of each of the plurality of multimedia contents.
Ziman discloses obtaining the order, based on the profile information including at least one of identification information of the user or information indicating browsing of each of the plurality of multimedia contents (Ziman, paragraph [0187];paragraph [0205]; Ziman discloses user interface elements for identifying and modifying information. ).
It would have been obvious for a person of ordinary skill in the art before the effective fling date of the claimed invention to use the interactive virtual world of Ziman with the mixed-reality space map creation device of Kim, Cappello, Bailey, and Wang to improve user interactions in a virtual space.
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 2019/0073832 A1) in view of Cappello et al. (Hereinafter, Cappello, US 2021/0183146 A1), Bailey et al. (Hereinafter, Bailey, US 20210/256769 A1), Wang et al. (Hereinafter, Wang, US 2020/0169586 A1), and further view of Wexler al. (Hereinafter, Wexler, US 2019/0332352 A1).
Per claim 9, Kim, Cappello, Bailey, and Wang disclose the wearable device of claim 1, but do not expressly disclose wherein the wearable device includes a microphone, wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device to: receive a sound signal indicating selection of the at least one multimedia content corresponding to the visual object, from the user, using the microphone, and identify the interaction based on receiving the sound signal.
Wexler discloses wherein the wearable device includes a microphone (Abstract; paragraph [0006], …The system may include a wearable camera configured to capture a plurality of images from an environment of the user; at least one microphone configured to capture sounds from an environment of the user; and at least one processor …. “), wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device processor is configured to: receive a sound signal indicating selection of the at least one multimedia content corresponding to the visual object, from the user, using the microphone, and identify the interaction based on receiving the sound signal(paragraph [0217], “ … Hearing interface device 1710 may include one or more speakers for providing audible feedback to user 100, a communication unit for receiving signals from another system, such as apparatus 110, microphones for detecting sounds in the environment of user 100, internal electronics, processors, memories, etc. Hearing interface device 1710 may correspond to feedback outputting unit 230 or may be separate from feedback outputting unit 230 and may be configured to receive signals from feedback outputting unit 230. “; paragraph [0218]; paragraph [0605]).
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the voice and visual signatures of Wexler with the mixed-reality space map creation device of Kim, Cappello, Bailey, and Wang to enhance one's interaction in his environment with feedback and other advanced functionality based on the analysis of captured image and audio data as suggested by Wexler (paragraph [0003]).
Per claim 18, Kim, Cappello, Bailey, and Wang disclose the method of the wearable device of claim 11, but do not expressly disclose wherein the wearable device includes a microphone, wherein the method further comprising: receiving a sound signal indicating selection of the at least one multimedia content corresponding to the visual object, from the user, using the microphone, and identifying the interaction, based on receiving the sound signal.
Wexler discloses wherein the wearable device includes a microphone (Abstract; paragraph [0006], …The system may include a wearable camera configured to capture a plurality of images from an environment of the user; at least one microphone configured to capture sounds from an environment of the user; and at least one processor …. “), wherein the processor is further configured to: receive a sound signal indicating selection of the at least one multimedia content corresponding to the visual object, from the user, using the microphone, and identify the interaction based on receiving the sound signal(paragraph [0217], “ … Hearing interface device 1710 may include one or more speakers for providing audible feedback to user 100, a communication unit for receiving signals from another system, such as apparatus 110, microphones for detecting sounds in the environment of user 100, internal electronics, processors, memories, etc. Hearing interface device 1710 may correspond to feedback outputting unit 230 or may be separate from feedback outputting unit 230 and may be configured to receive signals from feedback outputting unit 230. “; paragraph [0218]; paragraph [0605]).
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the voice and visual signatures of Wexler with the mixed-reality space map creation device of Kim, Cappello, Bailey, and Wang to enhance one's interaction in his environment with feedback and other advanced functionality based on the analysis of captured image and audio data as suggested by Wexler (paragraph [0003]).
Claim 10 is rejected under 35 U.S.C. 103 as being unpatentable over Kim (US 2019/0073832 A1) in view of Cappello et al. (Hereinafter, Cappello, US 2021/0183146 A1), Bailey et al. (Hereinafter, Bailey, US 20210/256769 A1), Wang et al. (Hereinafter, Wang, US 2020/0169586 A1), and further view of Evans (US 2019/0213796 A1).
Per claim 10, Kim, Cappello, Bailey, and Wang disclose the wearable device of claim 1, but does not expressly disclose wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device processor is configured to: identify a type of the virtual space, and obtain the visual object based on the type.
Evans discloses wherein the instructions, when executed by the at least one processor individually or collectively, cause the electronic device processor is configured to: identify a type of the virtual space(e.g., steps 810-820 as shown in Fig. 8; paragraph [0099-0100]), and obtain the visual object based on the type(e.g., steps 830-840 as shown in Fig. 8; paragraph [0101-0104]).
It would have been obvious for a person of ordinary skill in the art before the
effective fling date of the claimed invention to use the virtual reality portal
of Evans with the mixed-reality space map creation device of Kim, Cappello, Bailey, and Wang to facilitate real interactions with an identified electronic device as suggested by Evans (See Abstract).
Response to Arguments
Applicant’s arguments with respect to claims 1, 11, and 20 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to DARRIN HOPE whose telephone number is (571)270-5079. The examiner can normally be reached Mon-Thr - 6:45-4:15, Fri - 6:45-3:15, Alt. Fri Off.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Stephen S Hong can be reached at (571)272-4124. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
DARRIN HOPE
Examiner
Art Unit 2178
/STEPHEN S HONG/Supervisory Patent Examiner, Art Unit 2178
Prosecution Insights