DETAILED ACTION
*Note in the following document:
1. Texts in italic bold format are limitations quoted either directly or conceptually from claims/descriptions disclosed in the instant application.
2. Texts in regular italic format are quoted directly from cited reference or Applicant’s arguments.
3. Texts with underlining are added by the Examiner for emphasis.
4. Texts with
5. Acronym “PHOSITA” stands for “Person Having Ordinary Skill In The Art”.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 7 is objected to because of the following informalities:
Claim 7 recites a pick-up image of a front of the first user. Suggest replacing a pick-up image with “a picked-up image” since the phrase picked-up image is cited in multiple claims (or it is suggested replacing “picked-up image” with “pick-up image” or other phrase to make the claim language consistence).
Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 11-12 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 11/12 recite(s) by imaging by the imaging device, in the second real space. There is insufficient antecedent basis for this limitation in the claim. For the compact prosecution, the Examiner interprets the cited the imaging device is intended to be “an imaging device” since the phrase “imaging device” is not recited before in those claims.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2 and 4-12 are rejected under 35 U.S.C. 103 as being unpatentable over Matsuno et al. (US 2019/0278295 A1) in view of and Torres et al. (US 2024/0012491 A1).
Regarding Claim 1, Matsuno discloses a control device (Fig.3-6) for controlling a display device to be worn by a first user in a first real space (Fig.22), the control device comprising one or more processors and/or circuitry (Fig.3) configured to:
PNG
media_image1.png
467
727
media_image1.png
Greyscale
execute control processing of controlling the display device such that a virtual object disposed at a position in the first real space corresponding to a position of a second user in a second real space is displayed (Fig.22: virtual object 47 and [0276]: The operation computer 10 then reproduces the second space 52 based on the results of measurements with the virtual space computation module 103; thereby virtualizes the virtual space 53. Thereby, an avatar 47, wearing the power assist suit 300, of the person 46 appears in the virtual space 53).
Matsuno discloses the second user is detectable from a picked-up image acquired by an imaging device through imaging in the second real space and the virtual object is movable in the first real space ([0275]-[0276]: Color-depth sensors 39A-39C are placed in the second space 52. The person 46 wears the power assist suit 300 and stands up in the second space 52. The color-depth sensors 39A-39C make RGBD measurements of the person 46 and objects therearound, and send the results of measurements to the operation computer 10. The operation computer 10 receives the result of measurement from each of the color-depth sensors 39A-39C. The operation computer 10 then reproduces the second space 52 based on the results of measurements with the virtual space computation module 103; thereby virtualizes the virtual space 53. Thereby, an avatar 47, wearing the power assist suit 300, of the person 46 appears in the virtual space 53).
In addition, Matsuno further discloses making a virtual object from the second space movable in the virtual merged space ([0047]: The second space 52 includes a robot 3. The robot 3 directly handles a variety of objects in the second space 52 and [0200]: The solution module 107 enables the robot 3 to overcome the obstacle or to step back from the obstacle).
But Matsuno fails to disclose controlling the display device such that a range-display object indicating a range, in which the virtual object is movable in the first real space, is displayed; wherein the range, in which the virtual object is movable in the first real space, corresponds to an effective range; and the effective range is a range in which the second user is movable in the second real space.
However Torres discloses A method and a communication device for enabling execution of a virtual activity in a physical environment, wherein development of the virtual activity is presented to a user via at least one user interface of a communication device, the method including: … capturing and consolidating sensor data associated with the physical environment and virtual data associated with the virtual activity; generating at least two mutually dependent zones in the physical environment based on the captured and consolidated data and the trajectory of the communication device, and presenting an updated scene of the virtual activity, including at least parts of the zones via at least one of the user interfaces of the communication device (Abstract). Torres discloses the suggested method introduce multiple risk levels, which are presented to a player 100a,100b in a VR scene, seen by the player 100a,100b, in the form of clearly distinguishable zones, where the appearance and accuracy of the zones may depend on e.g. the specific one or more obstacle under consideration, as well as on the reliability and/or availability of sensors ([0029]) and the communication device 900a can be caused to dynamically adapt at least one of the zones, based on updated captured and consolidated data and an updated trajectory of the communication device 900a and present at least part of the at least one of the zones via at least one of the user interfaces of the communication device 900a during execution of the virtual activity ([0089]). Torres further discloses the four different zones can be safe zone, danger zone, buffer zone, and warning zone as shown in Fig.6a ([0070]: A user entering a buffer zone 620 from a safe zone 600 may trigger one or more groups or categories of sensors to capture sensor data at an increased frequency, in order to enable a faster and more precise reaction to the user, later entering a danger zone 610 from the buffer zone 620. [0071]: Optionally, yet another zone, here referred to as a warning zone 630 or alerting zone, may be applied, where such a zone has the main purpose of indicating to the user that a potential hazard may be about to arise in the zone, but that there is no imminent hazard to be in that zone).
PNG
media_image2.png
294
434
media_image2.png
Greyscale
Therefore it would have been obvious to a PHOSITA before the effective filing date to incorporate the teaching of Torres into that of Matsuno and to include the limitation of controlling the display device such that a range-display object indicating a range (Torres Fig.6a: safe zone 600), in which the virtual object is movable in the first real space, are displayed; wherein the range, in which the virtual object is movable in the first real space, corresponds to an effective range; and the effective range is a range in which the second user is movable in the second real space in order to make a user of a VR application aware of obstacles that can appear as a potential hazard to the user, when the user is executing the VR application in a dynamic, unpredictable environment, as well as to provide such awareness with a minimum of impact on the VR activity as suggested by Torres ([0006]).
Regarding Claim 2, Torres further teaches or suggests wherein in the control processing, in the range-display object, each position in a range, in which the virtual object is movable in the first real space, is displayed in a display form according to detection accuracy of a state of the second user at a position in the second real space corresponding to the position concerned ([0029]: More specifically, the suggested method introduce multiple risk levels, which are presented to a player 100a,100b in a VR scene, seen by the player 100a,100b, in the form of clearly distinguishable zones, where the appearance and accuracy of the zones may depend on e.g. the specific one or more obstacle under consideration, as well as on the reliability and/or availability of sensors). The same reason to combine as that of Claim 1 is applied.
Regarding Claim 4, Torres teaches or suggests wherein the one or more processors and/or circuitry is further configured to: execute acquisition processing of acquiring range information indicating the effective range; and generation processing of generating the range-display object on a basis of the range information (Fig.3 and [0045]-[0049] and [0029]). The same reason to combine as that of Claim 1 is applied.
PNG
media_image3.png
735
323
media_image3.png
Greyscale
Regarding Claim 4, Matsuno further teaches or suggests wherein in the acquisition processing, state information indicating a state of the second user including the position of the second user is further acquired; and in the generation processing, the virtual object is generated on a basis of the state information ([0275]-[0276]: Color-depth sensors 39A-39C are placed in the second space 52. The person 46 wears the power assist suit 300 and stands up in the second space 52. The color-depth sensors 39A-39C make RGBD measurements of the person 46 and objects therearound, and send the results of measurements to the operation computer 10. The operation computer 10 receives the result of measurement from each of the color-depth sensors 39A-39C. The operation computer 10 then reproduces the second space 52 based on the results of measurements with the virtual space computation module 103; thereby virtualizes the virtual space 53. Thereby, an avatar 47, wearing the power assist suit 300, of the person 46 appears in the virtual space 53. [0283]: When the person 46 swings, the head-mounted display 12 displays an image of the avatar 47 swinging).
Regarding Claim 6, Matsuno modified by Torres further teaches or suggests wherein
the display device is a display device capable of visual recognition of an outside through a display (Torres Fig.2 and [0037]: In the example of FIG. 2 there is also another instruction, indicating the closest way to exit the zone to the player, here with an arrow 220 and the text “10 m to a safe zone in this direction”); and
PNG
media_image4.png
323
496
media_image4.png
Greyscale
in the control processing, the display device is controlled such that the virtual object and the range-display object are displayed on the display (See Torres Fig.2: notice zones divided by zone markings 210a/b: within 210a is danger zone, between 210a and 210b is buffer zone and outside 210b is safe zone). The same reason to combine as that of Claim 1 is applied.
Regarding Claim 7, Matsuno discloses generating a virtual space which combines both real space of a first user’s and the second user’s as shown in Fig.2/22. Torres discloses displaying different zones according to dangerous levels in virtual environment. Therefore the combination of Matsuno and Torres teaches or suggests in the control processing, the display device is controlled such that an image, in which the virtual object and the range-display object are synthesized with a pick-up image of a front of the first user, is displayed. The same reason to combine as that of Claim 1 is applied.
Regarding Claim 8, Matsuno modified by Torres further teaches or suggests wherein in the control processing, the display device is controlled such that an image of a virtual space is displayed (Matsuno Fig.22), and a third image indicating a range corresponding to a range, in which the first user is movable in the first real space, and indicating a range in an image of the virtual space is further displayed (Torres Fig.2 and Fig.6a). The same reason to combine as that of Claim 1 is applied.
Regarding Claim 9, Claim 9 is/are similar to Claim 1 except in the format of system. Matsuno further discloses the system comprises an imaging device that acquires a picked-up image by imaging the second real space ([0275]: Color-depth sensors 39A-39C are placed in the second space 52. The person 46 wears the power assist suit 300 and stands up in the second space 52. The color-depth sensors 39A-39C make RGBD measurements of the person 46 and objects therearound, and send the results of measurements to the operation computer 10). Therefore the same reason(s) for rejection is/are applied to Claim 1 is/are also applied to Claim 9.
Regarding Claim 10, Matsuno modified by Torres further discloses the system comprising one or more processors and/or circuitry, configured to execute detection processing of detecting the effective range on a basis of the picked-up image (Matsuno Fig.3-5: e.g. the CPU 10a in Fig.3, [0275]: Color-depth sensors 39A-39C are placed in the second space 52. The person 46 wears the power assist suit 300 and stands up in the second space 52. The color-depth sensors 39A-39C make RGBD measurements of the person 46 and objects therearound, and send the results of measurements to the operation computer 10. [0283]: a When the person 46 swings, the head-mounted display 12 displays an image of the avatar 47 swinging. Torres [0029]: More specifically, the suggested method introduce multiple risk levels, which are presented to a player 100a,100b in a VR scene, seen by the player 100a,100b, in the form of clearly distinguishable zones, where the appearance and accuracy of the zones may depend on e.g. the specific one or more obstacle under consideration, as well as on the reliability and/or availability of sensors. Fig.3-4 and also refer to [0045]-[0060] for explanation regarding Fig.3/4). The same reason to combine as that of Claim 1 is applied.
Regarding Claim 11, Claim 11 is/are similar to Claim 1 except in the format of method. Therefore the same reason(s) for rejection is/are applied to Claim 1 is/are also applied to Claim 11.
Regarding Claim 12, Claim 12 is/are similar to Claim 1 except in the format of non-transitory computer readable medium. Therefore the same reason(s) for rejection is/are applied to Claim 1 is/are also applied to Claim 12.
Allowable Subject Matter
Claim 3 is objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
The following is a statement of reasons for the indication of allowable subject matter:
Prior art, either individually or in combination, fails to disclose or render obviousness the limitation of detection accuracy of a state of the second user is accuracy based on at least either one of a range reflected in the picked-up image of a whole body of the second user and a size of the second user in the picked-up image as claimed in dependent claim 3. The closest prior art, Torres et al. (US 20204/0012491 A1), discloses displaying danger zones in virtual space. However, it fails to disclose above cited limitation.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Abe et al. (US 2015/0356788 A1) discloses merging two real space environments into a virtual space as shown in Fig.13 and [0169].
PNG
media_image5.png
383
586
media_image5.png
Greyscale
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YINGCHUN HE whose telephone number is (571)270-7218. The examiner can normally be reached M-F 8:00-5:00 MT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao M Wu can be reached at 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YINGCHUN HE/Primary Examiner, Art Unit 2613
Prosecution Insights