Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Use of indicates a limitation is not explicitly disclosed by the reference alone.
Claim(s) 1-6, 8-13, 15-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Urbach (US 2017/0061700)
Claim 1
Urbach discloses a virtual reality server for generating a virtual user device in a virtual environment that is associated with a real-world user device in a real-world environment (Urbach, ¶ 37, 44: “In the virtual reality mode, a user's view of the real world is replaced by the display generated by a processor associated with the lenses or viewing screen of the device…Virtual object 104 can be generated either directly by the wearable computing device 108 or it may be a rendering received from another remote device…the remote device can be a server”), the virtual reality server comprising:
PNG
media_image1.png
531
408
media_image1.png
Greyscale
a memory configured to store one or more real-world images of the real-world user device in the real-world environment (Urbach, ¶ 50: “In some embodiments, the pre-processing module 250 can be a software of an ‘app’ stored in a local storage of the computing device 206 and executable by a processor comprised within the computing device 206.”); and
a processor configured to execute instructions stored in the memory, the instructions, when executed by the processor (Urbach, ¶ 2: “detecting, by a processor in communication with a first display device, presence of a real-world object comprising a marker on a surface thereof”), configuring the processor to:
extract one or more tracking markers from the one or more real-world images of the real-world user device (Urbach, abstract, ¶ 10, 48: “Using sensor and camera data of the first display device, a real-world object with a marker on its surface is identified…a real-world object comprising a marker on a surface…Examples of such computing devices which can be used as real-world objects can comprise without limitation smartphones, tablets,”),
estimate a pose of the real-world user device from the one or more tracking markers (Urbach, ¶ 2: “The processor identifies position and orientation of the real-world object in real 3D space relative to a user's eyes and renders a virtual object positioned and oriented in a virtual 3D space relative to the marker.”),
decode the one or more tracking markers to identify one or more virtual images that are associated with the virtual user device (Urbach, ¶ 61: “As mentioned supra, the marker 210 can be an image, a QR code, a bar code and the like. Hence, the marker 210 can be configured so that it encodes information associated with the particular virtual object 204 to be generated. In some embodiments, the pre-processing module 250 can be configured to display different markers each of which can each encode information corresponding to a particular virtual object.”), and
render the one or more virtual images to have the pose of the real-world user device to generate the virtual user device for display in the virtual environment (Urbach, ¶ 48: “FIG. 2 is an illustration 200 that shows generation of a virtual object 204 with respect to a marker 210 on a touch-sensitive surface 212 in accordance with some embodiments.”).
Urbach does not explicitly reference decoding. However, as disclosed by Urbach: “the marker 210 can be configured so that it encodes information associated with the particular virtual object 204 to be generated”. Furthermore, “when the camera of the wearable device 108 scans the QR code, the appropriate rendering is retrieved by the wearable device 108 from the server (not shown) and projected on to the model 402.”).
Therefore when configured according to an embodiment using encoding, and before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to use decoding. The motivation would have been to retrieve the relevant information associated with the marker in order to retrieve the appropriate virtual object.
Claim 2
Urbach discloses wherein the instructions, when executed by the processor, configure the processor to receive the one or more real-world images of the real-world user device from a virtual reality display device that is to display the virtual user device in the virtual environment (Urbach, abstract, ¶ 58: “Using sensor and camera data of the first display device, a real-world object with a marker on its surface is identified…when the camera of the wearable device 108 scans the QR code, the appropriate rendering is retrieved by the wearable device 108 from the server (not shown) and projected on to the model 402.”).
Claim 3
Urbach discloses wherein the instructions, when executed by the processor, configure the processor to receive the one or more real-world images of the real-world user device from the virtual reality display device in real-time (Urbach, ¶ 58: “when the camera of the wearable device 108 scans the QR code, the appropriate rendering is retrieved by the wearable device 108 from the server (not shown) and projected on to the model 402.”).
Claim 4
Urbach discloses wherein the pose of the real-world user device comprises a position, an orientation, or a motion of the real-world user device in the real-world environment, and wherein the instructions, when executed by the processor, configure the processor to estimate the position, the orientation, or the motion of the real-world user device from the one or more tracking markers (Urbach, ¶ 2, 65: “he processor identifies position and orientation of the real-world object in real 3D space relative to a user's eyes and renders a virtual object positioned and oriented in a virtual 3D space relative to the marker… The data from the wearable device camera is also analyzed to identify a marker, its position and orientation relative to the real-world object 106 that comprises the marker thereon. As discussed supra, one or more virtual objects can be generated and/or manipulated relative to the marker.”)
Claim 5
Urbach discloses wherein the instructions, when executed by the processor, configure the processor to estimate the pose of the real-world user device in relative to a virtual reality display device that is to display the virtual user device in the virtual environment (Urbach, ¶ 49: “precise positioning or tracking of the computing device 206 relative to the wearable device 108”)
Claim 6
Urbach discloses wherein the instructions, when executed by the processor, configure the processor to: decode the one or more tracking markers to recover one or more identifiers that have been encoded within the one or more tracking markers; and access the one or more virtual images that are associated with the one or more identifiers (Urbach, ¶ 58, 61: “when the camera of the wearable device 108 scans the QR code, the appropriate rendering is retrieved by the wearable device 108 from the server (not shown) and projected on to the model 402…As mentioned supra, the marker 210 can be an image, a QR code, a bar code and the like. Hence, the marker 210 can be configured so that it encodes information associated with the particular virtual object 204 to be generated. In some embodiments, the pre-processing module 250 can be configured to display different markers each of which can each encode information corresponding to a particular virtual object.”).
Claim 8
The same teachings and rationales in claim 1 are appliable to claim 8.
Claim 9
The same teachings and rationales in claim 2 are appliable to claim 9.
Claim 10
The same teachings and rationales in claim 3 are appliable to claim 10.
Claim 11
The same teachings and rationales in claim 4 are appliable to claim 11.
Claim 12
The same teachings and rationales in claim 5 are appliable to claim 12.
Claim 13
The same teachings and rationales in claim 6 are appliable to claim 13.
Claim 15
The same teachings and rationales in claim 1 are appliable to claim 15, with Urbach also disclosing a system for generating a virtual user device in a virtual environment that is associated with a real-world user device in a real-world environment, the system comprising: a virtual reality display device configured to capture one or more real-world images of the real-world user device in the real-world environment; and a virtual reality server configured to (e.g. HMD device, real world smartphone and server configuration (See claim 1))
Claim 16
The same teachings and rationales in claim 2 are appliable to claim 16.
Claim 17
The same teachings and rationales in claim 3 are appliable to claim 17.
Claim 18
The same teachings and rationales in claim 4 are appliable to claim 18.
Claim 19
The same teachings and rationales in claim 5 are appliable to claim 19.
Claim 20
The same teachings and rationales in claim 6 are appliable to claim 20.
Claim(s) 7, 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Urbach (US 2017/0061700) in view of Avola, A Practical Framework for the Development of Augmented Reality Applications by using ArUco Markers
Claim 7
Urbach does not explicitly disclose, but Avola discloses wherein the one or more tracking markers comprise one or more ArUco markers (Section 3.3: “ArUco4 markers(Figure1f)since they meet all our requirements in terms of robust ness, reliability and versatility (Garrido-Jurado et al., 2014). Moreover, their recognition is very fast with a low level of ambiguity in indoor environments.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to substitute ArUco markers for other types of visual markers such as QR codes.
The claimed invention differs from claim 7 because different visual markers are used in Rochford. The functions of visual markers were well known in the art, and one of ordinary skill in the art would have recognized that ArUco may be substituted for other camera detected markers. One of ordinary skill in the art could have made the substitution and the results would have been predictable because Urbach utilizes camera detectable markers which are interchangeable in this context.
Claim 14
The same teachings and rationales in claim 7 are appliable to claim 14.
Additional Prior Art
Additional prior art relevant to Applicant’s disclosure but not relied upon:
Rochford (US 2017/0249726) considers positional tracking for generating a virtual device image using markers:
PNG
media_image2.png
645
429
media_image2.png
Greyscale
Kaino (US 2016/0133058) considers associating markers with virtual content:
PNG
media_image3.png
404
441
media_image3.png
Greyscale
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN M GRAY whose telephone number is (571)272-4582. The examiner can normally be reached on Monday through Friday, 9:00am-5:30pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN M GRAY/Primary Examiner, Art Unit 2611
Prosecution Insights