Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim(s) 2objected to because of the following informalities:
Regarding claim(s) 2, the claim term virtualized physical object lacks an antecedent basis in claim 1. Examiner interprets virtualized physical as referring back to the virtual representation of the physical object from claim 1.,
Appropriate correction is required.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
Claim(s) 1-21 rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-20 of U.S. Patent No. 9,754,167 in view of Wong, Yoon, Osman and Da Silva
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘018 is broader in scope than ‘167. However, the corresponding additional features are disclosed by Wong and Osman (See rejection below for supporting paragraphs and rationales). Given the base invention in ‘167, before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider the additional features. As disclosed by Wong, in the context of virtual objects in a worn device, various tracking of relative movement of real world objects can be used to improve safety by alerting users to changes in relative distance. For the same reasons, the teachings of Osman would also be appliable because both references consider tracking the relative movement of real objects for the purposes of alerting the user.
Claim(s) 1-21 rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-20 of U.S. Patent No. 10,043,320 in view of Wong, Yoon, Osman and Da Silva
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘018 is broader in scope than ‘320. However, the corresponding additional features are disclosed by Wong and Osman (See rejection below for supporting paragraphs and rationales). Given the base invention in ‘320, before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider the additional features. As disclosed by Wong, in the context of virtual objects in a worn device, various tracking of relative movement of real world objects can be used to improve safety by alerting users to changes in relative distance. For the same reasons, the teachings of Osman would also be appliable because both references consider tracking the relative movement of real objects for the purposes of alerting the user.
Claim(s) 1-21 rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-20 of U.S. Patent No. 10,475,249 in view of Wong, Yoon, Osman and Da Silva
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘018 is broader in scope than ‘249. However, the corresponding additional features are disclosed by Wong and Osman (See rejection below for supporting paragraphs and rationales). Given the base invention in ‘249, before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider the additional features. As disclosed by Wong, in the context of virtual objects in a worn device, various tracking of relative movement of real world objects can be used to improve safety by alerting users to changes in relative distance. For the same reasons, the teachings of Osman would also be appliable because both references consider tracking the relative movement of real objects for the purposes of alerting the user.
Claim(s) 1-21 rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-20 of U.S. Patent No. 11,538,224 in view of Wong, Yoon, Osman and Da Silva
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘018 is broader in scope than ‘224. However, the corresponding additional features are disclosed by Wong and Osman (See rejection below for supporting paragraphs and rationales). Given the base invention in ‘224, before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider the additional features. As disclosed by Wong, in the context of virtual objects in a worn device, various tracking of relative movement of real world objects can be used to improve safety by alerting users to changes in relative distance. For the same reasons, the teachings of Osman would also be appliable because both references consider tracking the relative movement of real objects for the purposes of alerting the user.
Claim(s) 1-21 rejected on the ground of nonstatutory double patenting as being unpatentable over claim(s) 1-20 of U.S. Patent No. 12,125,157 in view of Wong, Yoon, Osman and Da Silva
Although the claims at issue are not identical, they are not patentably distinct from each other because ‘018 is broader in scope than ‘157. However, the corresponding additional features are disclosed by Wong and Osman (See rejection below for supporting paragraphs and rationales). Given the base invention in ‘157, before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider the additional features. As disclosed by Wong, in the context of virtual objects in a worn device, various tracking of relative movement of real world objects can be used to improve safety by alerting users to changes in relative distance. For the same reasons, the teachings of Osman would also be appliable because both references consider tracking the relative movement of real objects for the purposes of alerting the user.
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/process/file/efs/guidance/eTD-info-I.jsp.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Use of indicates a limitation is not explicitly disclosed by the reference alone.
Claim(s) 1-17, 19-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wong (US 2013/0335301) in view of Yoon (US 2011/0090252) and Osman (US 2014/0364212)
Claim 1
Examiner’s Interpretation:
Machine readable media can encompass forms of signal transmission media that falls outside of the four statutory categories of invention. MPEP 2106; citing In re Nuijten, 500 F.3d 1346, 84 USPQ2d 1495 (Fed. Cir. 2007). A claim whose BRI covers both statutory and non-statutory embodiments embraces subject matter that is not eligible for patent protection and therefore is directed to non-statutory subject matter. MPEP 2106.
Claims 1-18 as drafted recites non-transitory computer readable medium.
Because non-transitory without further definition excludes signal media, the broadest reasonable interpretation of the claimed medium in view of Applicant’s specification covers only eligible subject matter.
Claim Mapping:
Wong discloses a non-transitory computer readable medium impressed with instructions for detecting an imminent collision with a physical object during a virtual reality experience by a user wearing a virtual reality device (VRD) (Wong, ¶ 92: “HMD may implement method 600 in order to initiate a collision-avoidance action in response to a nearby object.”), the instructions when executed by one or more processors cause the processors to perform operations comprising:
using, by the VRD, (Wong, ¶ 99: “The wearable computer may use with various sensors or combinations of sensors to acquire the data that is used to initiate a collision-avoidance action. For example, some embodiments may utilize data from video cameras”) that is blocked, by the VRD, from being actually viewed, wherein there is no virtual representation of the physical object (e.g. Tree 710; Wong, ¶ 120: “FIG. 7A, including a physical object 710 near the display. As shown in FIG. 7A, the view of the physical object is substantially obstructed by virtual object 706 and virtual object 708”);
analyzing, by the VRD, multiple images to detect relative motion of the physical object that is blocked by the VRD, from being viewed by the user as the user approaches (Wong, ¶ 94, 99, 104: “relative movement of an object may also be considered to be the movement of the object relative to the wearer of such a wearable compute…The camera and wearable computer may transmit this estimated distance data to the computing system carrying out an object-detection process. Hence the object detection process may use the already determined distance as a basis for activating a collision-avoidance action…sensors may be employed, alone…”);
detecting that the user is approaching the physical object in dependence on (i) the identified proximity of the physical object (Wong, ¶ 99: “a. Determining Distance…utilize data from video cameras”) and (ii) the detected relative motion of the physical object wherein detecting the relative motion involves a capability of detecting motion of the physical object (Wong, ¶ 94, 99, 104: “relative movement of an object may also be considered to be the movement of the object relative to the wearer of such a wearable compute…The camera and wearable computer may transmit this estimated distance data to the computing system carrying out an object-detection process. Hence the object detection process may use the already determined distance as a basis for activating a collision-avoidance action…sensors may be employed, alone…”); and
responsive to detecting that the user is approaching the physical object, automatically generating an alert to be injected into a virtual reality experience of the user by generating data for display that represents the physical object in a virtual environment of the VRD (Wong, ¶¶ 32, 130: “To help avoid this danger, the wearable computer may be configured to detect approaching and/or nearby objects and alert the wearer of the potential risk….the user-interface has presented a new virtual object 718 to draw attention to the approaching physical object 710. This example new virtual object 718 is shown as a caution symbol and a border around the detected object”).
Wong does not disclose, but Yoon discloses a feature in an image of a sequence of captured images (Yoon, ¶ 12: “The image processing unit may further include a feature tracking unit to track the extracted feature points in a next image I.sub.t+1 acquired at a position to which the camera is moved.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects through different perspectives.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Wong as modified by Yoon does not explicitly disclose, but Osman discloses a VRD (Fig. 2A);
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to consider virtual reality.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence as the techniques applied to AR are equally appliable and Osman also suggests use of alerts due to real objects. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Claim 2
Wong discloses wherein the virtualized physical object comprises edges (Wong, ¶¶ 32, 130: “To help avoid this danger, the wearable computer may be configured to detect approaching and/or nearby objects and alert the wearer of the potential risk….the user-interface has presented a new virtual object 718 to draw attention to the approaching physical object 710. This example new virtual object 718 is shown as a caution symbol and a border around the detected object”).
Claim 3
Wong discloses wherein the edges are part of one or more silhouettes (an outline corresponds to a silhouette; Wong, ¶¶ 32, 130: “To help avoid this danger, the wearable computer may be configured to detect approaching and/or nearby objects and alert the wearer of the potential risk….the user-interface has presented a new virtual object 718 to draw attention to the approaching physical object 710. This example new virtual object 718 is shown as a caution symbol and a border around the detected object”)..
Claim 4
Wong discloses further including: analyzing with multiple images to detect relative motion between the physical object approached and the user; and responsive to detecting the physical object approached within a proximity threshold, automatically generating an alert to be injected into a virtual reality experience of the user (e.g. a purely camera based embodiment determining relative distance; (Wong, ¶ 94, 99, 104: “relative movement of an object may also be considered to be the movement of the object relative to the wearer of such a wearable compute…The camera and wearable computer may transmit this estimated distance data to the computing system carrying out an object-detection process. Hence the object detection process may use the already determined distance as a basis for activating a collision-avoidance action…sensors may be employed, alone…”)
Claim 5
Wong discloses further including adapting a threshold cutoff of the proximity threshold based on an environment (Wong, ¶ 95: “e wearable computer may adjust the threshold distance according to the relative movement and/or direction of a nearby object. For instance, a wearable computer may generally reduce the threshold distance at which collision avoidance is initiated as (a) the nearby object's relative velocity towards the wearable computer increases and/or (b) as the nearby object's trajectory indicates a collision is more likely, and vice versa”)
Claim 6
Wong discloses wherein the automatic generating of the alert further includes generating a physical warning feedback (Wong, ¶ 132: “As with presenting new virtual objects, a nonvisual alarm may be specific to the situation. For example, if an object is determined to be best avoided by a wearer moving left, an audio alert may include a voice saying, "Move left". Additionally, non-visual cues could include physical cues such as vibration of the system or physical movement of display windows.”).
Claim 7
Wong discloses wherein the physical warning feedback is a visual feedback generated in a virtual environment of the VRD (Wong, ¶ 130: “ adding one or more virtual objects to highlight the detected physical object.”).
Claim 8
Wong discloses wherein the physical warning feedback is at least one of a haptic feedback, and an audio feedback (Wong, ¶ 132: “As with presenting new virtual objects, a nonvisual alarm may be specific to the situation. For example, if an object is determined to be best avoided by a wearer moving left, an audio alert may include a voice saying, "Move left". Additionally, non-visual cues could include physical cues such as vibration of the system or physical movement of display windows.”).
Claim 9
Wong discloses further including using a feature in the image to distinguish among different approaching physical objects in a real world environment (Wong, ¶ 131: “For example, new virtual object 718 from FIG. 7H includes a border around the detected object. If a different physical object were detected, this border would need to be reshaped to encompass this different object. If the detected physical object were outside of the field of view covered by the display window, a new virtual object indicating the direction of the detected object may be presented as part of a collision-avoidance action.”).
Claim 10
Wong does not disclose, but Yoon discloses further including using a feature in the image to distinguish among different perspectives of a particular approaching physical object from varying vantage points of a camera of the VRD (Yoon, ¶ 12: “The image processing unit may further include a feature tracking unit to track the extracted feature points in a next image I.sub.t+1 acquired at a position to which the camera is moved.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects through different perspectives.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Claim 11
Wong does not disclose, but Yoon discloses wherein the proximity of the physical object is identified by tracking a motion of the physical object from a first position in a first reference frame to a second position in a second reference frame and transforming the first position and the second position to a common reference frame that does not change as the VRD is repositioned (Yoon,
PNG
media_image1.png
507
604
media_image1.png
Greyscale
Fig. 6; ¶ 31: “FIG. 6 is a view showing a coordinate system on an object plane and an image plane and a projection relationship according to an embodiment;”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects through different perspectives.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Claim 12
Wong does not disclose, but Yoon discloses wherein the transforming of the first position and the second position of the physical object into the common reference frame further includes applying an affine transformation to at least one of the first reference frame and the second reference frame (see Equation 2; Yoon, ¶ 66: “The plane projective invariant is not varied if the plane is satisfied even when the position of the camera is varied”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects through different perspectives.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Claim 13
Wong does not disclose, but Yoon discloses wherein the common reference frame comprises a world reference frame (projection invariant world; Yoon, ¶ 66: “The plane projective invariant is not varied if the plane is satisfied even when the position of the camera is varied”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects through different perspectives.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Claim 14
The combined references disclose further including determining an orientation of the physical object at the first position with respect to the first reference frame and causing the display of the physical object accordingly (e.g. display of indicator objects corresponding to real object position by both Osman and Wong, tracking disclosed by Yoon, ¶ 12: “The image processing unit may further include a feature tracking unit to track the extracted feature points in a next image I.sub.t+1 acquired at a position to which the camera is moved.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects through different perspectives.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Claim 15
The combined references disclose further including determining an orientation of the physical object at the second position with respect to the second reference frame and causing the display of the physical object accordingly (e.g. display of indicator objects corresponding to real object position by both Osman and Wong, tracking disclosed by Yoon, ¶ 12: “The image processing unit may further include a feature tracking unit to track the extracted feature points in a next image I.sub.t+1 acquired at a position to which the camera is moved.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects through different perspectives.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Claim 16
The combined references disclose creating a virtual three-dimensional (3D) map of a real world environment surrounding the user of the VRD, using the VRD and at least one camera coupled thereto; using features in some images of the sequence of images to detect a position or shape of the physical object and incorporating the physical object detected as 3D constraints in the virtual 3D map, wherein the physical object detected as 3D constraints represents an actual position or shape of the physical object; and responsive to detecting an approaching 3D constraint within a proximity threshold of the VRD, automatically signaling the VRD to generate the alert to the user (Both Wong and Osman track in 3D space; Yoon Fig. 8 disclosing 3D constraints)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects in 3D space as claimed.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Claim 17
Wong discloses wherein the automatic generating of the alert further includes at least one of:
generating data that represents the physical object approached;
entering a pass through mode that interrupts the virtual reality experience and substitutes a live feed of the real world environment in a virtual environment of the VRD; and
generating a physical warning feedback, including: a haptic feedback; an audio message generated by the VRD; or a visual feedback generated in a virtual environment of the VRD (Wong, ¶ 32: “Exemplary alert techniques may likewise take on many forms, including de-emphasizing virtual objects, presenting audio alerts, and/or displaying new virtual objects to highlight the physical object, among others.”)
Claim 19
Examiner’s Interpretation:
Machine readable media can encompass forms of signal transmission media that falls outside of the four statutory categories of invention. MPEP 2106; citing In re Nuijten, 500 F.3d 1346, 84 USPQ2d 1495 (Fed. Cir. 2007). A claim whose BRI covers both statutory and non-statutory embodiments embraces subject matter that is not eligible for patent protection and therefore is directed to non-statutory subject matter. MPEP 2106.
Claim 19 as drafted recites non-transitory computer readable medium.
Because non-transitory without further definition excludes signal media, the broadest reasonable interpretation of the claimed medium in view of Applicant’s specification covers only eligible subject matter.
Claim Mapping:
The same teachings and rationales in claim 1 are appliable with relative object tracking disclosing the different limitation of an object approaching the user.
Claim 20
The same teachings and rationales in claim 1 are appliable to claim 20.
Claim 21
The same teachings and rationales in claim 1 are applicable to claim 21, with Wong disclosing a system comprising a non-transitory computer readable medium including instructions for detecting an imminent collision with a physical object during a virtual reality experience by a user wearing a virtual reality device (VRD), the instructions when executed by one or more processors cause the processors to perform operations comprising (Wong, ¶ 92: “HMD may implement method 600 in order to initiate a collision-avoidance action in response to a nearby object.”):
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Wong (US 2013/0335301) in view of Yoon (US 2011/0090252), Osman (US 2014/0364212) and Da Silva (US 2012/0170800)
Claim 18
Wong does not disclose, but Da Silva discloses wherein the using of the feature further includes detecting at least one difference between the feature between a first time t0 and a second time t1 using at least one of: a vector field flow; and a probabilistic flow (da Silva, ¶ 23: “At a high-level, the process described herein executes an optical flow algorithm to identify features in a previously captured frame of video and tracks the movement of those features in the current video frame. In the process, displacement vectors may be calculated for each feature and may be subsequently used to estimate an intermediate, intra-frame position for each feature.”)
Before the effective filing date of this application, it would have been obvious to one of ordinary skill in the art to track objects over time using optical flow.
One of ordinary skill in the art would have motivation to consider object the technique to track moving objects in a video sequence. One of ordinary skill in the art would have had a reasonable expectation of success because Wong suggests uses of a camera to track relative position of objects.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RYAN M GRAY whose telephone number is (571)272-4582. The examiner can normally be reached on Monday through Friday, 9:00am-5:30pm (EST).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached on (571)272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see https://ppair-my.uspto.gov/pair/PrivatePair. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RYAN M GRAY/Primary Examiner, Art Unit 2611
Prosecution Insights