DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1 and 10 are rejected on the ground of nonstatutory obviousness-type double patenting as being unpatentable over claims 1 and 9, respectively, of Patent No. 12,026,806 in view of Chen et al. (US Pub. 2014/0035901), hereinafter Chen.
Regarding independent claims 1 and 10 of the instant application, although the conflicting claims are not identical, they are not patentably distinct from each other because they are generic to all that is recited in patent 12,026,806 claims 1 and 9, respectively, in view of Chen; that is, claims 1 and 10 are anticipated by claims 1 and 9, respectively, of the patent 12,026,806, in view of Chen, and are therefore an obvious variant thereof. This will be described in reference to the table below; emphasis has been added in bold to the corresponding elements.
Instant Application 18/758475
Patent 12,026,806
Claim 1:A system configured to utilize non-stationary living entities as markers for virtual content viewed in an augmented reality environment, the system comprising:
a display device configured to superimpose images of virtual content over a real-world view of a user to create a visual effect of the augmented reality environment being present in a real world, wherein the real-world view of the user is an outlook of the real world from the point of view of the user;
electronic storage that stores information related to the virtual content;
one or more physical computer processors configured by computer-readable instructions to:
receive user input, via a user interface, that includes virtual content information that define individual virtual content items, wherein the virtual content information defines one or more colors, a shape, one or more sounds, a size, and/or a position for the individual virtual content items;
store the virtual content information in the electronic storage;
obtain an image of a field of view of the real-world view of the user visible via the display device, wherein the field of view is the area that comprises the real-world view of the user;
detect multiple linkage points for the non-stationary living entities;
determine individual actions of the non-stationary living entities as indicated by an arrangement of the multiple linkage points;
obtain the virtual content information from the electronic storage based on the actions;
determine images of the virtual content items to be displayed in the augmented reality environment for the non-stationary living entities based on at least the virtual content information and the field of view; and
cause the images of the virtual content items to be displayed in the augmented reality environment so that the images of the virtual content items are superimposed over the non-stationary living entities in the real-world view of the user such that the user views the images of the virtual content items in motion in accordance with the actions of the non-stationary living entities in the augmented reality environment via the display device.
Claim 1:
A system configured to utilize living entities as markers for virtual content viewed in an augmented reality environment, the system comprising:
a display device configured to superimpose images of visual virtual content over a real-world view of a user to create a visual effect of the augmented reality environment being present in a real world, wherein the real-world view of the user is an outlook of the real world from the point of view of the user;
electronic storage that stores information related to the visual virtual content;
one or more physical computer processors configured by computer-readable instructions to:
receive user input, via a user interface, that includes virtual content information that define individual virtual content items, wherein the virtual content information defines one or more colors, a shape, one or more sounds, a size, and/or a position for the individual virtual content items;
store the virtual content information in the electronic storage;
obtain an image of a field of view of the real-world view of the user visible via the display device, wherein the field of view is the area that comprises the real-world view of the user;
receive signals from transponders of living entities within the field of view, wherein the signals from the transponders indicate locations of the transponders and includes triggering information that causes images of virtual content items to be rendered in the augmented reality environment based on the living entities;
detect multiple linkage points for the living entities based on the transponders, wherein the signals from the transponders define positions of the multiple linkage points with respect to the living entities that correlate to specific portions of the virtual content items;
obtain the virtual content information from the electronic storage based on the triggering information;
generate images of the virtual content items to be displayed in the augmented reality environment based on the transponders of the living entities, wherein the images of the virtual content items are generated based on at least on the virtual content information and the field of view; and
cause the images of the virtual content items to be displayed in the augmented reality environment so that the images of the virtual content items are superimposed over the living entities in the real-world view of the user such that the user views the images of the virtual content items in the augmented reality environment via the display device.
Claim 10:
A method of utilizing non-stationary living entities as markers for virtual content viewed in an augmented reality environment, the method comprising:
receiving user input, via a user interface, that includes virtual content information that define individual virtual content items, wherein the virtual content information defines one or more colors, a shape, one or more sounds, a size, and/or a position for the individual virtual content items;
storing the virtual content information in electronic storage;
obtaining an image of a field of view of a real-world view of a user that is visible via a display device, wherein the display device is configured to superimpose images of virtual content over the real-world view of the user to create a visual effect of the augmented reality environment being present in a real world, wherein the real-world view of the user is an outlook of the real world from the point of view of the user, wherein the field of view is the area that comprises the real-world view of the user;
detecting multiple linkage points for the non-stationary living entities;
determining individual actions of the non-stationary living entities as indicated by an arrangement of the multiple linkage points;
obtaining the virtual content information from the electronic storage based on the actions;
determining images of the visual virtual content items to be displayed in the augmented reality environment for the non-stationary living entities based on at least the virtual content information and the field of view; and
causing the images of the virtual content items to be displayed in the augmented reality environment so that the images of the virtual content items are superimposed over the non-stationary living entities in the real-world view of the user such that the user views the images of the virtual content items in motion in accordance with the actions of the non-stationary living entities in the augmented reality environment via the display device.
Claim 9: A method of utilizing living entities as markers for virtual content viewed in an augmented reality environment, the method comprising:
receiving user input, via a user interface, that includes virtual content information that define individual virtual content items, wherein the virtual content information defines one or more colors, a shape, one or more sounds, a size, and/or a position for the individual virtual content items;
storing the virtual content information in electronic storage;
obtaining an image of a field of view of a real-world view of a user that is visible via a display device, wherein the display device is configured to superimpose images of visual virtual content over the real-world view of the user to create a visual effect of the augmented reality environment being present in a real world, wherein the real-world view of the user is an outlook of the real world from the point of view of the user, wherein the field of view is the area that comprises the real-world view of the user;
receiving signals from transponders of living entities within the field of view, wherein the signals from the transponders indicate locations of the transponders and includes triggering information that causes images of virtual content items to be rendered in the augmented reality environment based on the living entities;
detecting multiple linkage points for the living entities based on the transponders, wherein the signals from the transponders define positions of the multiple linkage points with respect to the living entities that correlate to specific portions of the virtual content items;
obtaining the virtual content information from the electronic storage based on the triggering information;
generating images of the visual virtual content items to be displayed in the augmented reality environment based on the transponders of the living entities, wherein the images of the virtual content items are generated based on at least on the virtual content information and the field of view; and
causing the images of the virtual content items to be displayed in the augmented reality environment so that the images of the virtual content items are superimposed over the living entities in the real-world view of the user such that the user views the images of the virtual content items in the augmented reality environment via the display device.
As can be seen in the above table, the system and method claims of this application 18/758475 embody and perform the functions as presented in the system and method claims, respectively, of patent 12,026,806.
The system and method claims of patent 12,026,806 do not explicitly disclose determining individual actions of the non-stationary living entities as indicated by an arrangement of the multiple linkage points, as in claims 1 and 10.
However, Chen teaches utilizing non-stationary living entities as markers for virtual content viewed in an augmented reality environment (Paragraph [0022]; Paragraph [0035]), further comprising: determining individual actions of the non-stationary living entities as indicated by an arrangement of the multiple linkage points (Fig. 5; Paragraph [0022]: body tracking data defines positions of one or more points on a body and any type of body tracking data may be used which enables correspondences between the sensor data and nodes in the deformation graph to be determined; Paragraph [0057]: shown in FIG. 5, the node (or vertex) positions within the deformation graph are defined initially using sampling (block 502). These node positions may defined by traversing the vertices of the input mesh or by distributing nodes over the surface of the object and then using Poisson Disk sampling (which may also be referred to as Poisson Disk pattern generation). Where nodes are defined by traversing the vertices of the input mesh, the method involves selecting a region of surface (e.g. selecting a triangle and then a point inside the triangle), picking a radius based on the total area of surface and using dart throwing until a desired number of samples is reached). Chen teaches that this will allow for realistic avatars with realistic deformations (Paragraphs [0082]-[0083]). Therefore, it would have been obvious to one of ordinary skill in the art to have modified patent 12,026,806 with the features of above as taught by Chen so as to allow for realistic avatars with realistic deformations as presented by Chen.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 2, 8, 11, and 17 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claims 2, 8, 11, and 17 each recites the limitation "the transponders" in each of the clauses. There is insufficient antecedent basis for this limitation in the claim as no such transponder has been established. Appropriate correction and/or explanation is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3-7, 10, and 12-16 are rejected under 35 U.S.C. 103 as being unpatentable over Chen et al. (US Pub. 2014/0035901), hereinafter Chen, in view of Yasutake (US Pub. 2015/0371447).
Regarding claim 1, Chen discloses a system configured to utilize non-stationary living entities as markers for virtual content viewed in an augmented reality environment, the system comprising: a display device configured to superimpose images of virtual content over a real-world view of a user to create a visual effect of the augmented reality environment being present in a real world, wherein the real-world view of the user is an outlook of the real world from the point of view of the user (Fig. 3, item 320; Paragraph [0023]: A visual display may be provided to the user at this attachment stage so that they can position their body such that it approximates the shape of the object to be animated (e.g. such that their tracked skeleton approximately aligns with the object) and so the attachment of the skeleton to the graph is done in a more intuitive manner. Image 104 shows the overlapping of the skeleton 107 and the chair 108; Paragraph [0025]: Once the transformations have been computed on the deformation graph (in block 118), these transformations are applied to the input mesh and the corresponding motion (i.e. the animation) of the object is rendered for display to the user (block 120). Images 105 and 106 show two example images from an animation of the chair; Paragraph [0045]: generates the 3D geometry from the real-world 3D object and outputs the 3D geometry to both the Embed stage 202 and the Warp stage; Paragraph [0056]: When attaching the tracked skeleton to the deformation graph (in block 116), this may be performed automatically by adding point constraints in space. For example, each joint in the tracked skeleton (or each point in the body tracking data) may be made a constraint. In such an example, at attachment time, the position of each joint pl (or other point, where joints are not used) is stored along with Euclidean distances to the k-nearest nodes in the deformation graph. As the user moves, each joint (or point) moves to a new position ql and the graph is transformed such that the k-nearest nodes; Paragraph [0057]: presentation techniques may be used to make this GUI clearer or more intuitive, for example, by making the object partially transparent and/or showing the tracked skeleton in a bright color and/or by displaying skeleton overlaid over the object (or vice-versa). In some examples, the GUI is arranged such that the user appears to walk up to the object within the GUI); electronic storage that stores information related to the virtual content (Fig. 3; Paragraph [0031]: computer executable instructions may be provided using any computer-readable media that is accessible by computing based device 300); one or more physical computer processors configured by computer-readable instructions (Fig. 3) to: receive user input, via a user interface, that includes virtual content information that define individual virtual content items, wherein the virtual content information defines one or more colors, a shape, one or more sounds, a size, and/or a position for the individual virtual content items (Fig. 3; Paragraph [0022]: the animation method uses the body of the user as an input and body tracking data for the user is acquired using a sensor and received from the sensor (in block 114). Any suitable sensor may be used, including but not limited to, non-contact sensors such as camera-based systems (e.g. Kinect™, Wii™) and marker-based tracking systems (e.g. using Vicon™ markers) and contact-based sensors such as a multi-touch device. The body tracking data defines positions of one or more points on a body and any type of body tracking data may be used which enables correspondences between the sensor data and nodes in the deformation graph to be determined; Paragraph [0057]: which nodes in the graph are linked to each joint (e.g. through the point constraints described above) will depend on the position of the user when the tracked skeleton is obtained. As the user has freedom to adopt different body positions, the user can influence or directly control which parts of their body are mapped to particular parts of the object being animated. In order to assist the user in this, a graphical user interface (GUI) may be provided which provides visual feedback to the user, for example by displaying both the tracked skeleton and the object (or more particularly, the mesh of the object). Various presentation techniques may be used to make this GUI clearer or more intuitive, for example, by making the object partially transparent and/or showing the tracked skeleton in a bright color and/or by displaying skeleton overlaid over the object (or vice-versa). In some examples, the GUI is arranged such that the user appears to walk up to the object within the GUI; Paragraph [0056]: When attaching the tracked skeleton to the deformation graph (in block 116), this may be performed automatically by adding point constraints in space. For example, each joint in the tracked skeleton (or each point in the body tracking data) may be made a constraint. In such an example, at attachment time, the position of each joint pl (or other point, where joints are not used) is stored along with Euclidean distances to the k-nearest nodes in the deformation graph. As the user moves, each joint (or point) moves to a new position ql and the graph is transformed such that the k-nearest nodes); store the virtual content information in the electronic storage (Fig. 3; Paragraph [0056]: When attaching the tracked skeleton to the deformation graph (in block 116), this may be performed automatically by adding point constraints in space. For example, each joint in the tracked skeleton (or each point in the body tracking data) may be made a constraint. In such an example, at attachment time, the position of each joint pl (or other point, where joints are not used) is stored along with Euclidean distances to the k-nearest nodes in the deformation graph. As the user moves, each joint (or point) moves to a new position ql and the graph is transformed such that the k-nearest nodes); obtain an image of a field of view of the real-world view of the user visible via the display device (Paragraph [0023]: enable the body of the user to be used as an input, this tracked skeleton (as defined by the body tracking data received in block 114) is attached to the deformation graph (block 116). As the deformation graph is a representation of the input mesh, the tracked skeleton may also be considered to be attached to the mesh. The attachment of the skeleton to the graph may be performed automatically without user input or in response to voice commands from the user, as is described in more detail below. A visual display may be provided to the user at this attachment stage so that they can position their body such that it approximates the shape of the object to be animated (e.g. such that their tracked skeleton approximately aligns with the object) and so the attachment of the skeleton to the graph is done in a more intuitive manner. Image 104 shows the overlapping of the skeleton 107 and the chair; Paragraph [0025]: Once the transformations have been computed on the deformation graph (in block 118), these transformations are applied to the input mesh and the corresponding motion (i.e. the animation) of the object is rendered for display to the user (block 120). Images 105 and 106 show two example images from an animation of the chair. The first image 105 shows the chair walking and the second image 106 shows the chair jumping. These images are generated, using the method described above, when the user walks and jumps respectively); detect multiple linkage points for the non-stationary living entities (Paragraph [0022]: the animation method uses the body of the user as an input and body tracking data for the user is acquired using a sensor and received from the sensor (in block 114). Any suitable sensor may be used, including but not limited to, non-contact sensors such as camera-based systems (e.g. Kinect™, Wii™) and marker-based tracking systems (e.g. using Vicon™ markers) and contact-based sensors such as a multi-touch device. The body tracking data defines positions of one or more points on a body and any type of body tracking data may be used which enables correspondences between the sensor data and nodes in the deformation graph to be determined); determine individual actions of the non-stationary living entities as indicated by an arrangement of the multiple linkage points (Fig. 5; Paragraph [0022]: body tracking data defines positions of one or more points on a body and any type of body tracking data may be used which enables correspondences between the sensor data and nodes in the deformation graph to be determined; Paragraph [0057]: shown in FIG. 5, the node (or vertex) positions within the deformation graph are defined initially using sampling (block 502). These node positions may defined by traversing the vertices of the input mesh or by distributing nodes over the surface of the object and then using Poisson Disk sampling (which may also be referred to as Poisson Disk pattern generation). Where nodes are defined by traversing the vertices of the input mesh, the method involves selecting a region of surface (e.g. selecting a triangle and then a point inside the triangle), picking a radius based on the total area of surface and using dart throwing until a desired number of samples is reached); obtain the virtual content information from the electronic storage based on the actions (Paragraph [0079]: the tracked skeleton may be used to animate more than one object (e.g. a user's legs may be attached to a deformation graph of a chair and the same user's arms may be attached to a deformation graph of a desk lamp). Alternatively, only part of a user's body may be mapped to the deformation graph of an object (e.g. only a user's arms or only their fingers). Similarly, an entire skeleton may be used to animate only part of an object. Consequently, the attachment stage described above may be used to attach all or part of one or more skeletons to all or part of one or more deformation graphs. In some examples, the attachment process may occur in multiple phases, for example a first phase may be used to attach a first part of a skeleton to a first part of a deformation graph and a second phase may be used to attach a second part of a skeleton to a second part of a deformation graph and the user may be able to move in between phases; Paragraph [0080]: objects being animated are non-humanoid objects where such objects may be inanimate objects (such as items of furniture or other household items) or animate objects (such as animals). In some examples, however, the objects being animated may be humans, such that, for example, one user can possess the avatar for another user. Furthermore, whilst the skeletons which are tracked and attached to the deformation graph are, in most examples, human skeletons, (i.e. the skeleton(s) of the user(s)), the methods are also applicable to scenarios where the tracked skeleton belongs to an animal and it is the animal skeleton that is attached to the deformation graph generated from the input mesh); determine images of the virtual content items to be displayed in the augmented reality environment for the non-stationary living entities based on at least the virtual content information and the field of view (Paragraph [0023]: enable the body of the user to be used as an input, this tracked skeleton (as defined by the body tracking data received in block 114) is attached to the deformation graph (block 116). As the deformation graph is a representation of the input mesh, the tracked skeleton may also be considered to be attached to the mesh. The attachment of the skeleton to the graph may be performed automatically without user input or in response to voice commands from the user, as is described in more detail below. A visual display may be provided to the user at this attachment stage so that they can position their body such that it approximates the shape of the object to be animated (e.g. such that their tracked skeleton approximately aligns with the object) and so the attachment of the skeleton to the graph is done in a more intuitive manner. Image 104 shows the overlapping of the skeleton 107 and the chair; Paragraph [0025]: Once the transformations have been computed on the deformation graph (in block 118), these transformations are applied to the input mesh and the corresponding motion (i.e. the animation) of the object is rendered for display to the user (block 120). Images 105 and 106 show two example images from an animation of the chair. The first image 105 shows the chair walking and the second image 106 shows the chair jumping. These images are generated, using the method described above, when the user walks and jumps respectively).
Chen does not explicitly disclose wherein the field of view is the area that comprises the real-world view of the user; and cause the images of the virtual content items to be displayed in the augmented reality environment so that the images of the virtual content items are superimposed over the non-stationary living entities in the real-world view of the user such that the user views the images of the virtual content items in motion in accordance with the actions of the non-stationary living entities in the augmented reality environment via the display device.
However, Yasutake teaches an augmented reality system including superimposing animated objects on living entities (Abstract), further comprising wherein the field of view is the area that comprises the real-world view of the user (Paragraph [0048]: an AR based environment for a first group of users who are physically located at the substantially same or similar location, where the AR based environment is a real world environment including interactive virtual objects (e.g., 3D AR objects); and (ii) an AV based environment for a second group of users who are physically located at different locations from the first group of users, where the AR based environment is a virtual reality environment including interactive virtual objects. In some embodiments, the AR based environment includes the first group of users as real persons, as well as virtual objects related to (e.g., representing, controlled by, manipulated by, etc.) the second group of users. In some embodiments, the AV based environment includes virtual objects related to (e.g., representing) the first group of users, as well as those virtual objects related to the second group of users); and cause the images of the virtual content items to be displayed in the augmented reality environment so that the images of the virtual content items are superimposed over the non-stationary living entities in the real-world view of the user such that the user views the images of the virtual content items in motion in accordance with the actions of the non-stationary living entities in the augmented reality environment via the display device (Fig. 4D; Paragraph [0049]: Each user of the hybrid reality environment can interact with each other user through the one or more server devices. In some embodiments, each user from the first group of users can, within the AR based environment, interact with each other user from the first group of users in a face-to-face manner (i.e., the two users are physically at the same location in the real world and interacting with each other); and interact with each virtual object related to the second group of users. On the other hand, each user from the second group of users can control or manipulate a corresponding virtual object related to that user to, within the AV based environment, interact with each virtual object related to the first group of users and the virtual object related to each other user from the second group of users (i.e., the two virtual objects are virtually at the same location in the virtual world and interacting with each other, controlled or manipulated by the corresponding users). In such a way, each user from the first group of users or the second group of users can physically or virtually interact with each other user from the first group of users or the second group of users; Paragraph [0080]: FIG. 4D is a block diagram illustrating functions performed by an AR application in connection with the schematic illustrations of FIGS. 4A-4C. Instructions for such an AR application can be stored in a memory of a computer device (e.g., a mobile device, a smart phone, etc.) of a user, and performed by a processor of that computer device. As shown in FIG. 4D, a 3D video camera installed at the computer device (e.g., at a rear side of a mobile device) can be used to capture the light from the subject (i.e., the real person), and convert, in a real-time manner, collected raw data into 3D location data in accordance with the coordinate system of set at the computer device. The AR application can also overlay the 3D AR creature (i.e., the AR tiger) in a scene of the AR based environment). Yasutake teaches that this will allow for hybrid reality environments and interactions in real time between users (Abstract; Paragraph [0060]). Therefore, it would have been obvious to one of ordinary skill in the art to have modified Chen with the features of above as taught by Yasutake so as to allow for hybrid reality environments and interactions in real time between users as presented by Yasutake.
Regarding claim 3, Chen, in view of Yasutake teaches the system of claim 1, Chen discloses wherein the virtual content items comprise an animation associated with the actions (Paragraph [0025]: transformations are applied to the input mesh and the corresponding motion (i.e. the animation) of the object is rendered for display to the user (block 120). Images 105 and 106 show two example images from an animation of the chair. The first image 105 shows the chair walking and the second image 106 shows the chair jumping. These images are generated, using the method described above, when the user walks and jumps respectively).
Regarding claim 4, Chen, in view of Yasutake teaches the system of claim 1, Chen discloses wherein the multiple positions of each of the multiple linkage points in the real world defines the reference frame of the virtual content items with respect to the real world and the non-stationary living entities (Fig. 5; Paragraph [0022]: body tracking data defines positions of one or more points on a body and any type of body tracking data may be used which enables correspondences between the sensor data and nodes in the deformation graph to be determined; Paragraph [0041]: shown in the first flow diagram 400 in FIG. 4, this example method of generating an input mesh, comprises three stages (blocks 401-403). The 3D reconstruction stage (block 401) estimates the 6-DoF (Degree of Freedom) pose of the moving Kinect™ camera while the user scans the object with the camera, and fuses depth data continuously into a regular 3D voxel grid data structure which may be stored on a GPU. Surface data is encoded implicitly into voxels as signed distances, truncated to a predefined region around the surface, with new values integrated using a weighted running average. The global pose of the moving depth camera is predicted using point-plane ICP (Iterative Closest Point), and drift is mitigated by aligning the current raw depth map with the accumulated model (instead of the previous raw frame). The system produces a 3D volumetric reconstruction of the scene accumulated from the moving depth camera; Paragraphs [0044]-[0048]: meshing stage (block 403) automatically extracts and triangulates the desired foreground isosurface stored implicitly in the voxel grid. A geometric isosurface is extracted from the foreground labeled volumetric dataset using a GPU-based marching cubes algorithm. For each voxel, the signed distance value at its eight corners is computed. The algorithm uses these computed signed distances as a lookup to produce the correct polygon at the specific voxel…distance metric which is used in performing the Poisson Disk sampling may be a Euclidean distance metric; however, this can miss areas of the surface with high curvature and result in artifacts where semantically unrelated parts are linked. Consequently, an alternative distance metric may be used in sampling: a 5D orientation-aware distance metric. Where this metric is used, the sampling may be referred to as `orientation-aware sampling).
Regarding claim 5, Chen, in view of Yasutake teaches the system of claim 1, Chen discloses wherein the images of the virtual content items are generated based further on multiple positions of the display device in the real world and the multiple positions of each of the multiple linkage points in the real world (Fig. 4; Paragraph [0041]: shown in the first flow diagram 400 in FIG. 4, this example method of generating an input mesh, comprises three stages (blocks 401-403). The 3D reconstruction stage (block 401) estimates the 6-DoF (Degree of Freedom) pose of the moving Kinect™ camera while the user scans the object with the camera, and fuses depth data continuously into a regular 3D voxel grid data structure which may be stored on a GPU. Surface data is encoded implicitly into voxels as signed distances, truncated to a predefined region around the surface, with new values integrated using a weighted running average. The global pose of the moving depth camera is predicted using point-plane ICP (Iterative Closest Point), and drift is mitigated by aligning the current raw depth map with the accumulated model (instead of the previous raw frame). The system produces a 3D volumetric reconstruction of the scene accumulated from the moving depth camera; Paragraphs [0044]-[0048]: meshing stage (block 403) automatically extracts and triangulates the desired foreground isosurface stored implicitly in the voxel grid. A geometric isosurface is extracted from the foreground labeled volumetric dataset using a GPU-based marching cubes algorithm. For each voxel, the signed distance value at its eight corners is computed. The algorithm uses these computed signed distances as a lookup to produce the correct polygon at the specific voxel…distance metric which is used in performing the Poisson Disk sampling may be a Euclidean distance metric; however, this can miss areas of the surface with high curvature and result in artifacts where semantically unrelated parts are linked. Consequently, an alternative distance metric may be used in sampling: a 5D orientation-aware distance metric. Where this metric is used, the sampling may be referred to as `orientation-aware sampling).
Regarding claim 6, Chen, in view of Yasutake teaches the system of claim 1, Chen discloses wherein the images of the virtual content items are generated based further on a size of the arrangement of the multiple linkage points within the field of view of the user (Paragraph [0039]: the input mesh may be downloaded from the internet or may be generated by scanning the object (e.g. using the same sensor 314 which is also used in tracking the user's motion). The methods described herein can use arbitrary meshes, and these meshes need not be complete and may be polygon soups (including incomplete polygon soups), triangle meshes, point clouds, volumetric data, watertight 3D models, etc. In various examples, the input mesh may be generated from a real-world non-human (e.g. inanimate) object of reasonable physical size and surface reflectance).
Regarding claim 7, Chen, in view of Yasutake teaches the system of claim 1, Chen discloses wherein the field of view is defined based on location information and orientation information, the location information indicating at least a current location associated with the display device, and the orientation information indicating at least a pitch angle, a roll angle, and a yaw angle associated with the display device (Fig. 4; Paragraph [0041]: shown in the first flow diagram 400 in FIG. 4, this example method of generating an input mesh, comprises three stages (blocks 401-403). The 3D reconstruction stage (block 401) estimates the 6-DoF (Degree of Freedom) pose of the moving Kinect™ camera while the user scans the object with the camera, and fuses depth data continuously into a regular 3D voxel grid data structure which may be stored on a GPU).
Regarding claim 10, the limitations of this claim substantially correspond to the limitations of claim 1; thus they are rejected on similar grounds.
Regarding claim 12, the limitations of this claim substantially correspond to the limitations of claim 3; thus they are rejected on similar grounds.
Regarding claim 13, the limitations of this claim substantially correspond to the limitations of claim 4; thus they are rejected on similar grounds.
Regarding claim 14, the limitations of this claim substantially correspond to the limitations of claim 5; thus they are rejected on similar grounds.
Regarding claim 15, the limitations of this claim substantially correspond to the limitations of claim 6; thus they are rejected on similar grounds.
Regarding claim 16, the limitations of this claim substantially correspond to the limitations of claim 7; thus they are rejected on similar grounds.
Claims 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Chen, in view of Yasutake, and further in view of Haitani et al. (US Patent 10664903), hereinafter Haitani.
Regarding claim 9, Chen, in view of Yasutake teaches the system of claim 1, wherein determining the images of the virtual content items to be displayed in the augmented reality environment for the non-stationary living entities.
Chen, in view of Yasutake does not explicitly disclose employing machine learning on the actions of the non-stationary living entities.
However, Haitani teaches an augmented reality system including superimposing animated objects on living entities (Abstract; Column 6), further comprising employing machine learning on the actions of the non-stationary living entities (Column 10, lines 1-31: objects or portions thereof expressed within imaging data may be associated with a label or labels according to one or more machine-learning classifiers, algorithms or techniques, including but not limited to nearest neighbor methods or analyses, artificial neural networks, factorization methods or techniques, K-means clustering analyses or techniques, similarity measures such as log likelihood similarities or cosine similarities, latent Dirichlet allocations or other topic models, or latent semantic analyses). Haitani teaches that this will provide an enhanced user interface (Column 10). Therefore, it would have been obvious to one of ordinary skill in the art to have modified Chen, in view of Yasutake with the features of above as taught by Haitani so as to allow for enhanced user interface as presented by Haitani.
Regarding claim 18, the limitations of this claim substantially correspond to the limitations of claim 9; thus they are rejected on similar grounds.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MATTHEW D SALVUCCI whose telephone number is (571)270-5748. The examiner can normally be reached M-F: 7:30-4:00PT.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, XIAO WU can be reached at (571) 272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MATTHEW SALVUCCI/Primary Examiner, Art Unit 2613