DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is responsive to the amendment received 09/17/2025.
In the response to the Non-Final Office Action 04/17/2025, the applicant states that claims 1, 10-15, and 20 have been amended. Claims 1-20 are pending in current application.
Claims 1, 10-15, and 20 have been amended. In summary, claims 1-20 are pending in current application.
Response to Arguments
Applicant's arguments filed 09/17/2025 have been fully considered.
Regarding to double patenting rejection, the applicant requests that the examiner reconsiders this non-statutory double patenting rejection when at least one independent claim of this application has been allowed or noted as being allowable over the cited references. The amendment does not overcome the double patenting rejection. The examiner maintains the double patenting rejection.
Regarding to 35 U.S.C 101 rejection, the amendment has cured the basis of 35 U.S.C 101 rejection. Therefore, the 35 U.S.C 101 rejection is hereby withdrawn.
Regarding to 35 U.S.C 112 (b) rejection, the amendment has cured the basis of 35 U.S.C 112 (b) rejection. Therefore, the 35 U.S.C 112 (b) rejection of claims 1, 5, 10, 14, 15, and 20 is hereby withdrawn.
Regarding to claim 1, the applicant argues that cited arts fail to disclose or suggest "determining a projection type of a plurality of projection types, for a projection for a user interaction with multiple objects and prior to the user interaction with the multiple objects" and "selecting the projection type of the plurality of projection types based on a computed distance between the user and the interaction plane." The arguments have been fully considered, but they are not persuasive. The examiner cannot concur with the applicant for following reasons:
Faaborg discloses “determining a projection type of a plurality of projection types, for a projection for a user interaction with multiple objects and prior to the user interaction with the multiple objects, by”. For example, in paragraph in paragraph[0004], Faaborg generating a far field display of a plurality of virtual objects; Faaborg further teaches selecting the far field display. In paragraph [0020], Faaborg teaches the electronic device emits and projects a ray or beam; Faaborg further teaches ray is determined; Faaborg further more teaches the ray and beam are a plurality of projection types. In paragraph [0023], Faaborg teaches a beam or ray emitted by the handheld electronic device 102 may pointed at a particular object in the virtual world to select the object. In paragraph [0030], Faaborg teaches emitting a ray or beam; Faaborg further teaches the ray and beam are a plurality of projection types. In paragraph [0032], Faaborg teaches the GT type interaction includes a ray-based interaction; select and manipulate objects by rays in the virtual world; Faaborg further teaches ray is determined. In paragraph [0039], Faaborg teaches a ray, or pointer beam; Faaborg further teaches a ray, and pointer beam are two projection types. In paragraph [0043], Faaborg teaches a point of a ray and beam emitted by the handheld electronic device 102 are different projection types. In Fig. 12A-G and paragraph [0053], Faaborg teaches the user interface is presented to the user as if on a wall at a relatively far distance; the far field display of the list 20 of objects; Faaborg further teaches determining a ray projection type to select multiple objects with gazing and prior to select objects as illustrated in Fig. 12A-G.
Powderly discloses “determining a projection type of a plurality of projection types”. For example, in paragraph [0095], Powderly teaches the ray casting techniques includes casting thin, pencil rays with substantially little transverse width or casting rays with substantial transverse width, e.g., cones or frustums. In paragraph [0106], Powderly teaches cone casting with a single ray may also be referred to as ray casting; Powderly further teaches cone casting and ray casting are two different projection types. In paragraph [0138], Powderly teaches the change in aperture can cause the casting shape to be a cone or ray; Powderly further teaches the smallest aperture size is zero which reduces the cone casting to a single ray casting; a single ray casting is ray casting; Powderly further more teaches Cone casting and ray casting are two different type casting.
Faaborg further discloses “selecting the projection type of a plurality of projection types”. For example, in paragraph [0020], Faaborg teaches the electronic device emits and projects a ray or beam; Faaborg further teaches the ray is determined; the ray and beam are a plurality of projection types. In paragraph [0030], Faaborg teaches emitting a ray or beam. In paragraph [0032], Faaborg teaches the GT type interaction includes a ray-based interaction; Faaborg further teaches selecting and manipulate objects by rays in the virtual world; Faaborg further more teaches ray is determined; in addition, Faaborg teaches the ray and beam are a plurality of projection types. In paragraph [0039], Faaborg teaches a ray, or pointer beam; Faaborg further teaches a ray, and pointer beam are two projection types. In paragraph [0041], Faaborg teaches a length of the beam. In Fig. 4B and paragraph [0050], Faaborg teaches a distance between an object and the user in the virtual world; Faaborg further teaches selecting a head gaze or an eye gaze. In paragraph [0051], Faaborg teaches determining the objects at a relatively far distance from the user in the virtual world. In Fig. 12A-G and paragraph [0053], Faaborg teaches determining a projection type, i.e., a ray, to select multiple objects as illustrated in Fig. 12A-G; a distance is calculated as 8 feet; Faaborg further teaches calculating a distance of approximately greater than 1.5 feet.
Powderly further discloses “selecting the projection type of a plurality of projection types based on a computed distance between the user and the interaction plane”. For example, in paragraph [0095], Powderly teaches the ray casting techniques includes casting thin, pencil rays with substantially little transverse width or casting rays with substantial transverse width, e.g., cones or frustums. In paragraph [0106], Powderly teaches cone casting with a single ray may also be referred to as ray casting; Powderly further teaches cone casting and ray casting are two different projection types. In paragraph [0108], Powderly teaches casting a ray and casting a cone. In paragraph [0129], Powderly teaches Cone casting is selected and used to increase precision when those objects are located at a distance where small amounts of movement from the user could translate to large movements of the ray. In paragraph [0138], Powderly teaches the change in aperture can cause the casting shape to be a cone or ray; Powderly further teaches the smallest aperture size is zero which reduces the cone casting to a single ray casting; a single ray casting is ray casting; Powderly further more teaches Cone casting and ray casting are two different type casting. In paragraph [0148-0149] and [0161-0162], Powderly teaches the aperture of the cone is adjusted based on the distance.
Claims 10 and 15 are not allowable due to the similar reasons as discussed above.
Claims 2-9, 11-14, and 16-19 are not allowable due to the similar reasons as discussed above.
Double Patenting
The nonstatutory double patenting rejection is based on a judicially created doctrine grounded in public policy (a policy reflected in the statute) so as to prevent the unjustified or improper timewise extension of the “right to exclude” granted by a patent and to prevent possible harassment by multiple assignees. A nonstatutory double patenting rejection is appropriate where the conflicting claims are not identical, but at least one examined application claim is not patentably distinct from the reference claim(s) because the examined application claim is either anticipated by, or would have been obvious over, the reference claim(s). See, e.g., In re Berg, 140 F.3d 1428, 46 USPQ2d 1226 (Fed. Cir. 1998); In re Goodman, 11 F.3d 1046, 29 USPQ2d 2010 (Fed. Cir. 1993); In re Longi, 759 F.2d 887, 225 USPQ 645 (Fed. Cir. 1985); In re Van Ornum, 686 F.2d 937, 214 USPQ 761 (CCPA 1982); In re Vogel, 422 F.2d 438, 164 USPQ 619 (CCPA 1970); In re Thorington, 418 F.2d 528, 163 USPQ 644 (CCPA 1969).
A timely filed terminal disclaimer in compliance with 37 CFR 1.321(c) or 1.321(d) may be used to overcome an actual or provisional rejection based on nonstatutory double patenting provided the reference application or patent either is shown to be commonly owned with the examined application, or claims an invention made as a result of activities undertaken within the scope of a joint research agreement. See MPEP § 717.02 for applications subject to examination under the first inventor to file provisions of the AIA as explained in MPEP § 2159. See MPEP § 2146 et seq. for applications not subject to examination under the first inventor to file provisions of the AIA . A terminal disclaimer must be signed in compliance with 37 CFR 1.321(b).
The filing of a terminal disclaimer by itself is not a complete reply to a nonstatutory double patenting (NSDP) rejection. A complete reply requires that the terminal disclaimer be accompanied by a reply requesting reconsideration of the prior Office action. Even where the NSDP rejection is provisional the reply must be complete. See MPEP § 804, subsection I.B.1. For a reply to a non-final Office action, see 37 CFR 1.111(a). For a reply to final Office action, see 37 CFR 1.113(c). A request for reconsideration while not provided for in 37 CFR 1.113(c) may be filed after final for consideration. See MPEP §§ 706.07(e) and 714.13.
The USPTO Internet website contains terminal disclaimer forms which may be used. Please visit www.uspto.gov/patent/patents-forms. The actual filing date of the application in which the form is filed determines what form (e.g., PTO/SB/25, PTO/SB/26, PTO/AIA /25, or PTO/AIA /26) should be used. A web-based eTerminal Disclaimer may be filled out completely online using web-screens. An eTerminal Disclaimer that meets all requirements is auto-processed and approved immediately upon submission. For more information about eTerminal Disclaimers, refer to www.uspto.gov/patents/apply/applying-online/eterminal-disclaimer.
Claims 1-20 are rejected on the ground of nonstatutory double patenting as being unpatentable over claims 1-20 of U.S. Patent No. US 11947111 B2 in view of Faaborg (US 20170060230 A1).
Application 18/587677
Claim 1
U.S. Patent No. US 11947111 B2
Claim 1
1. A method for interacting in an artificial reality environment, the method comprising:
1. A method for interacting in an artificial reality environment, the method comprising:
determining a projection type, for a projection for a user interaction with multiple objects and prior to the user interaction with the multiple objects, by:
determining a projection type for a projection to select multiple objects, prior to the selection of those multiple objects, by:
determining a control point and casting direction based on one or more tracked positions of one or more body parts of a user;
identifying an interaction plane of a gaze of a user,
identifying an interaction plane of a gaze of the user,
wherein the interaction plane is a viewing plane, on which the gaze of the user is focused prior to the user interaction,
wherein the interaction plane is a viewing plane, on which the gaze of the user is focused prior to the selecting of the multiple objects,
that is A) distant from the user in a 3D environment and B) is identified based on the gaze, by the user, directed toward a location between the multiple objects;
that is A) distant from the user in a 3D environment and B) intersects with a line formed by the gaze of the user, wherein the interaction plane is identified based on an interaction, at a location between the multiple objects, by the user via the gaze without selecting the multiple objects;
computing a distance between the user and the interaction plane prior to the selecting of the multiple objects; and
selecting the projection type based on the computed distance between the user and the interaction plane;
selecting the projection type based on the computed distance between the user and the interaction plane;
generating a projection, of the selected projection type; and
generating a projection, of the selected projection type, that begins at the control point and that extends along the casting direction; and
causing the user interaction with the multiple objects using the generated projection and based on one or more tracked positions of one or more body parts of the user.
identifying the multiple objects based on determining which objects at least partially intersect with at least part of the projection, either simultaneously or at different times.
2. The method of claim 1, wherein the projection type is equivalent to a cylinder type, specifying that the projection includes a cylinder that extends, beginning at a control point determined based on the one or more tracked positions of the one or more body parts of the user, outward from the user.
2. The method of claim 1, wherein the projection type is equivalent to a cylinder type, specifying that the projection includes a cylinder that extends, beginning at the control point, outward from the user and is centered on the casting direction.
Claims 3-20
Claims 3-20
The claims 1-20 of U.S. Patent No. US 11947111 B2 fails to explicitly disclose “causing the user interaction with the multiple objects using the generated projection and based on one or more tracked positions of one or more body parts of the user”.
In same field of endeavor, Faaborg (US 20170060230 A1) discloses “causing the user interaction with the multiple objects using the generated projection and based on one or more tracked positions of one or more body parts of the user”. For example, in paragraph [0020], Faaborg teaches eye directional gaze; Faaborg further teaches detecting and tracking the user's eye gaze. In paragraph [0023], Faaborg teaches user eye gaze is monitored and tracked; Faaborg further teaches movement of the object selected by the eye gaze. In paragraph [0026], Faaborg teaches defining a line of sight from the user's eye to an object; Faaborg further teaches the detected gaze is processed; Faaborg further more teaches eye is a control point. In Fig. 6A-D and paragraph [0037], Faaborg teaches a user shifts focus of a ray emitted by a handheld electronic device 102. In Fig. 12A-G and paragraph [0053], Faaborg teaches the user interface is presented to the user as if on a wall at a relatively far distance and the user's eye gaze intersects the user interface; Faaborg further teaches the far field display of the list 20 of objects; Faaborg further more teaches determining a projection type to select multiple objects as illustrated in Fig. 12A-G. The motivation for combining Faaborg with the claims 1-20 of U.S. Patent No. US 11947111 B2 would have been to select and to manipulate objects in the virtual world; to determine a projection type and to select multiple objects as illustrated in Fig. 12A-G as taught by Faaborg in Fig. 12A-G and paragraphs [0032], and [0053-0054].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 4-6, 10, 13-16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Faaborg (US 20170060230 A1) and in view of Powderly (US 20170287225 A1).
Regarding to claim 1 (Currently Amended), Faaborg discloses a method for interacting in an artificial reality environment (Fig. 1; [0006]: a virtual reality system; [0020]: a 3D virtual environment; HMD; [0027]: a virtual reality environment; Fig. 12A-E; [0053-0054]: draw the list of objects closer to the user; select an object 20A from the list 20 of objects 20A), the method comprising:
determining a projection type of a plurality of projection types, for a projection for a user interaction with multiple objects and prior to the user interaction with the multiple objects, by ([0004]: generate a far field display of a plurality of virtual objects; select the far field display; [0020]: the electronic device emits and projects a ray or beam; ray is determined; the ray and beam are a plurality of projection types; [0030]: emit a ray or beam; the ray and beam are a plurality of projection types; [0032]: the GT type interaction includes a ray-based interaction; select and manipulate objects by rays in the virtual world; ray is determined; [0039]: a ray, or pointer beam; a ray, and pointer beam are two projection types; [0043]: a point of a ray and beam emitted by the handheld electronic device 102 are different projection types; Fig. 12A-G; [0053]: the user interface is presented to the user as if on a wall at a relatively far distance; the far field display of the list 20 of objects; determine a ray projection type to select multiple objects with gazing and prior to select objects as illustrated in Fig. 12A-G):
identifying an interaction plane of a gaze of a user ([0004]: generate a far field display of a plurality of virtual objects; select the far field display; [0020]: translate the detected eye and/or head gaze into a corresponding interaction in the 3D virtual immersive experience; to allow the user to interact with the 3D virtual immersive experience generated by the HMD; [0023]: the user's gaze directed to an object in the virtual world may correspond to selection of that object; Fig. 12B; [0053]: the user's eye gaze intersects the user interface; the user's focus is established on the list 20 of objects 20A; Fig. 12C; [0054]: facilitate user selection, interaction and manipulation of the objects;
PNG
media_image1.png
334
486
media_image1.png
Greyscale
), wherein the interaction plane is a viewing plane, on which the gaze of the user is focused prior to the user interaction ([0004]: generate a far field display of a plurality of virtual objects; select the far field display; Fig. 12A-G; [0053]: the user interface is presented to the user as if on a wall at a relatively far distance; the far field display of the list 20 of objects; determine a projection type to select multiple objects and prior to select objects as illustrated in Fig. 12A-G; the user's gaze intersects the user interface as shown in FIG. 12B, the user's focus may be established on the list 20 of objects 20A; once focus is established on the list 20 of objects 20A, a trigger action may trigger dynamic switching to select objects; Fig. 12C; [0054]: facilitate user selection, interaction, and manipulation of the objects;
PNG
media_image1.png
334
486
media_image1.png
Greyscale
), that is A) distant from the user in a 3D environment and B) is identified based on the gaze, by the user, directed toward a location between the multiple objects ([0020]: the 3D virtual environment and interact with the 3D virtual environment through various different types of inputs; Fig. 12A-G; [0053]: the user interface is presented to the user as if on a wall at a relatively far distance; the far field display of the list 20 of objects; determine a projection type to select multiple objects and prior to select objects as illustrated in Fig. 12A-G; as the user's gaze intersects the user interface, and in particular, the list 20 of objects 20A, as shown in FIG. 12B, the user's focus may be established on the list 20 of objects 20A as described above;
PNG
media_image2.png
288
520
media_image2.png
Greyscale
);
selecting the projection type of a plurality of projection types ([0020]: the electronic device emits and projects a ray or beam; the ray is determined; the ray and beam are a plurality of projection types; [0030]: emit a ray or beam; [0032]: the GT type interaction includes a ray-based interaction; select and manipulate objects by rays in the virtual world; ray is determined; the ray and beam are a plurality of projection types; [0039]: a ray, or pointer beam; a ray, and pointer beam are two projection types; [0041]: a length of the beam; Fig. 4B; [0050]: a distance between an object and the user in the virtual world; select a head gaze or an eye gaze; [0051]: determine the objects at a relatively far distance from the user in the virtual world; Fig. 12A-G; [0053]: determine a projection type, i.e., a ray, to select multiple objects as illustrated in Fig. 12A-G; a distance is calculated as 8 feet; calculate a distance of approximately greater than 1.5 feet);
generating a projection, of the selected projection type ([0023]: a beam or ray emitted by the handheld electronic device 102 may pointe at a particular object in the virtual world to select the object; [0026]: define a line of sight from the user's eye to an object; ray begins at the eye and extends to the object; Fig. 6A-D; [0037]: a user shifts focus of a ray emitted by a the handheld electronic device 102; a ray is projected from the device, i.e. a control point, to object as illustrated in Fig. 6A-6D; Fig. 12B; Fig. 12C; [0053]: the user's eye gaze intersects the user interface; a ray is projected from user’s eye to list of objects as illustrated in Fig. 12B and Fig. 12C); and
causing the user interaction with the multiple objects using the generated projection and based on one or more tracked positions of one or more body parts of the user ([0020]: the electronic device emits and projects a ray or beam; ray is determined; detect and track the user's eye gaze; [0023]: user eye gaze is monitored and tracked; movement of the object selected by the eye gaze; [0026]: define a line of sight from the user's eye to an object; the detected gaze is processed; eye is a control point; [0030]: emit a ray or beam; [0032]: the GT type interaction includes a ray-based interaction; select and manipulate objects by rays in the virtual world; ray is determined; Fig. 6A-D; [0037]: a user shifts focus of a ray emitted by a the handheld electronic device 102; Fig. 12B; Fig. 12C; [0053]: the user's eye gaze intersects the user interface; Fig. 12A-G; [0053]: the user interface is presented to the user as if on a wall at a relatively far distance; the far field display of the list 20 of objects; determine a projection type, i.e., a ray, to select multiple objects as illustrated in Fig. 12A-G; as the user's gaze intersects the user interface, and in particular, the list 20 of objects 20A, as shown in FIG. 12B, the user's focus may be established on the list 20 of objects 20A as described above;
PNG
media_image2.png
288
520
media_image2.png
Greyscale
; a trigger action may trigger dynamic switching from the longer distance gaze input mode to the shorter distance reach/touch input mode; Fig. 12C; [0054]: granularity to facilitate user selection, interaction with and manipulation of the objects; Fig. 12E-F; [0057]: the selection of an object 20A from the list 20 of objects 20A, as well as the interaction with and manipulation of the selected object 20A is done with the user's hand;
PNG
media_image3.png
360
354
media_image3.png
Greyscale
).
Faaborg fails to explicitly disclose:
based on a computed distance between the user and the interaction plane.
In same field of endeavor, Powderly teaches:
determining a projection type of a plurality of projection types ([0095]: the ray casting techniques includes casting thin, pencil rays with substantially little transverse width or casting rays with substantial transverse width, e.g., cones or frustums; [0106]: Powderly teaches cone casting with a single ray may also be referred to as ray casting; Powderly further teaches cone casting and ray casting are two different projection types),
that is A) distant from the user in a 3D environment and B) is identified based on the gaze, by the user, directed toward a location between the multiple objects ([0118]: a location in the 3D space; a 3DOF hand-held controller; Fig. 12B; Fig. 12C; [0130]: cone casting on a group 1230 of objects, e.g. 1230a; object 1230a is an interaction plane as illustrated in Fig. 12B;
PNG
media_image4.png
382
737
media_image4.png
Greyscale
; [0144]: the user's 3D environment);
selecting the projection type of a plurality of projection types based on a computed distance between the user and the interaction plane ([0095]: the ray casting techniques includes casting thin, pencil rays with substantially little transverse width or casting rays with substantial transverse width, e.g., cones or frustums; [0106]: Powderly teaches cone casting with a single ray may also be referred to as ray casting; Powderly further teaches cone casting and ray casting are two different projection types; [0108]: Powderly teaches casting a ray and casting a cone. [0129]: Powderly teaches Cone casting is selected and used to increase precision when those objects are located at a distance where small amounts of movement from the user could translate to large movements of the ray; [0138]: the change in aperture can cause the casting shape to be a cone or ray; the smallest aperture size is zero which reduces the cone casting to a single ray casting; a single ray casting is ray casting. Cone casting and ray casting are two different type casting; [0148-0149]; [0161-0162]: the aperture of the cone is adjusted based on the distance).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Faaborg to include determining a projection type of a plurality of projection types, that is A) distant from the user in a 3D environment and B) is identified based on the gaze, by the user, directed toward a location between the multiple objects; selecting the projection type of a plurality of projection types based on a computed distance between the user and the interaction plane as taught by Powderly. The motivation for doing so would have been to permit a user interaction with objects in a field of regard (FOR) of a user; to interact with the objects, e.g., by selecting and moving objects, using poses or by actuating a user input device; to cause the casting shape to be a cone or ray as taught by Powderly in paragraphs [0004], [0028], and [0138].
Regarding to claim 4 (Original), Faaborg in view of Powderly discloses the method of claim 1,
wherein the projection type is equivalent to a cone type, specifying that the projection includes a cone (Powderly; Fig. 12A; [0108]: cone casting casts a conic volume 1220 with an adjustable aperture; a geometric cone has a proximal end 1228a and a distal end 1228b; [0109]: determine the direction 1224 of the cone using the user's head pose or eye pose); and
wherein a tip of the cone is at a control point determined based on the one or more tracked positions of the one or more body parts of the user and the cone extends, from the tip of the cone, outward from the user (Powderly; Fig. 12A; [0109]: determine the direction 1224 of the cone using the user's head pose or eye pose; [0110]: the proximal end 1228a of the cone 1220 is at the tip of the user's finger 1214; as the user points his finger to another location, the position of cone 1220 and the central ray 1224 are moved accordingly; [0117]: a central ray of the cone, etc.). The same motivation of claim 1 applies here.
Regarding to claim 5 (Currently Amended), Faaborg in view of Powderly discloses the method of claim 1,
wherein at least part of the projection is a cone (Powderly; Fig. 12A; [0108]: cone casting casts a conic volume 1220 with an adjustable aperture; a geometric cone has a proximal end 1228a and a distal end 1228b; [0109]: determine the direction 1224 of the cone using the user's head pose or eye pose ); and
wherein a width of a diameter of the base of the cone is based on the computed distance between the user and the interaction plane (Powderly; Fig. 3; [0044]: bring into focus objects at different distances along the z-axis; the width changes when depth distance changes as illustrated in Fig. 3; Fig. 12A; [0108]: a large aperture can correspond to a large diameter 1226 on the distal end 1228b of the cone 1220; the diameter changes from right to left along the cone direction as illustrated in Fig. 12A; Fig. 12B; Fig. 12C; [0130]: the object 1230a intersects with the cone 1220; automatically update the aperture based on contextual information; the contextual information includes distance; Fig. 13; [0148]: analyze the distance between the objects and the user; [0149]: adjust the size and width of the aperture based on the contextual information, i.e. distance; the large aperture size is correspond to a large diameter 1226 on the distal end 1228b of the cone 1220). The same motivation of claim 1 applies here.
Regarding to claim 6 (Original), Faaborg in view of Powderly discloses the method of claim 1, wherein the selecting the projection type is further based on an identified surface type of a target object (Powderly; [0117]: the distal end 1228b of the cone has any cross section, e.g., circular, oval, polygonal, or irregular; [0138]: the change in aperture causes the casting shape to be a cone or ray; the smallest aperture size is zero which reduces the cone to a single ray; Fig. 13; [0148-0149]: the aperture of the cone is adjusted based type of the objects, layout of the objects, location of the objects, size of the objects, density of the objects, and distance between the objects and the user on the distance; adjust the size of the aperture based on the contextual information; [0161-0162]). The same motivation of claim 1 applies here.
Regarding to claim 10 (Currently Amended), Faaborg discloses a non-transitory computer-readable storage medium storing instructions that, when executed by a computing system, cause the computing system to perform a process for interacting in an artificial reality environment, the process comprising (Fig. 1; [0006]: a virtual reality system; [0027]: a virtual reality environment; Fig. 3; [0029]: a processor 390; a memory 380; Fig. 3; [0030]: a processor 309; a controller 305 of the second electronic device 302, the controller 305 having access to a memory 308 and controlling overall operation of the second electronic device 302; Fig. 12A-E; [0053-0054]: draw the list of objects closer to the user; select an object 20A from the list 20 of objects 20A):
The rest limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject rest limitations of claim 10.
Regarding to claim 13 (Currently Amended), Faaborg in view of Powderly discloses the non-transitory computer-readable storage medium of claim 10,
The rest limitations are similar to claim limitations recited in claim 4. Therefore, same rational used to reject claim 4 is also used to reject rest limitations of claim 13.
Regarding to claim 14 (Currently Amended), Faaborg in view of Powderly discloses the non-transitory computer-readable storage medium of claim 10,
The rest limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject rest limitations of claim 14.
Regarding to claim 15 (Currently Amended), Faaborg discloses a non-transitory computing system for interacting in an artificial reality environment (Fig. 1; [0006]: a virtual reality system; [0027]: a virtual reality environment; Fig. 3; [0029]: a processor 390; a memory 380; Fig. 3; [0030]: a processor 309; a controller 305 of the second electronic device 302, the controller 305 having access to a memory 308 and controlling overall operation of the second electronic device 302; Fig. 12A-E; [0053-0054]: draw the list of objects closer to the user; select an object 20A from the list 20 of objects 20A), the computing system comprising:
one or more processors (Fig. 3; [0029]: a processor 390; Fig. 3; [0030]: a processor 309); and
one or more memories storing instructions that, when executed by the one or more processors, cause the computing system to perform a process comprising (Fig. 3; [0029]: a processor 390; a memory 380; Fig. 3; [0030]: a processor 309; the controller 305 having access to a memory 308 and controlling overall operation of the second electronic device 302; Fig. 14; [0062]: the processor 1402 processes instructions for execution within the computing device 1400, including instructions stored in the memory 1404):
the rest claim limitations are similar to claim limitations recited in claim 1. Therefore, same rational used to reject claim 1 is also used to reject claim 15.
Regarding to claim 16 (Original), Faaborg in view of Powderly discloses the computing system of claim 15,
the rest claim limitations are similar to claim limitations recited in claim 6. Therefore, same rational used to reject claim 6 is also used to reject claim 16.
Regarding to claim 20 (Currently Amended), Faaborg in view of Powderly discloses the computing system of claim 15,
the rest claim limitations are similar to claim limitations recited in claim 5. Therefore, same rational used to reject claim 5 is also used to reject claim 20.
Claims 2-3 and 11-12 are rejected under 35 U.S.C. 103 as being unpatentable over Faaborg (US 20170060230 A1) in view of Powderly (US 20170287225 A1), and further in view of Shohara (US 20210076091 A1).
Regarding to claim 2 (Original), Faaborg in view of Powderly discloses the method of claim 1,
Faaborg in view of Powderly fails to explicitly disclose:
wherein the projection type is equivalent to a cylinder type, specifying that the projection includes a cylinder that extends, beginning at a control point determined based on the one or more tracked positions of the one or more body parts of the user, outward from the user.
In same field of endeavor, Shohara teaches:
wherein the projection type is equivalent to a cylinder type, specifying that the projection includes a cylinder that extends, beginning at a control point determined based on the one or more tracked positions of the one or more body parts of the user, outward from the user (Fig. 1C; [0051]: the pointer P and a cursor are projected from the device in user’s hand, i.e. a control point, to the center of display using cylinder projection as illustrated in Fig. 1C; the direction indicated by the omnidirectional image capturing apparatus 110 and the position of the pointer P coincide; [0054]: a pointer P and a cursor; Fig. 2; [0068]: the user moves the pointer P on the image viewer operation screen S displayed on the display apparatus 180 by projecting pointer P from a device in a user hand to the display as illustrated in Fig. 2; [0095]: a projection method includes the eccentric cylinder projection method or the like; Fig. 14; [0165]: display of the pointer P displayed when its coordinates are initialize).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Faaborg in view of Powderly to include wherein the projection type is equivalent to a cylinder type, specifying that the projection includes a cylinder that extends, beginning at a control point determined based on the one or more tracked positions of the one or more body parts of the user, outward from the user as taught by Shohara. The motivation for doing so would have been to indicate a position in the operation screen at which a use operation is input; to obtain an omnidirectional image by synthesizing the two fisheye images based on the eccentric cylinder projection method as taught by Shohara in paragraphs [0054] and [0095].
Regarding to claim 3 (Original), Faaborg in view of Powderly discloses the method of claim 1,
Faaborg in view of Powderly fails to explicitly disclose:
wherein the projection type is equivalent to a line-and-sphere type, specifying that the projection includes at least part of a sphere that is at an end of a line that extends, beginning at a control point determined based on the one or more tracked positions of the one or more body parts of the user, outward from the user.
In same field of endeavor, Shohara teaches:
wherein the projection type is equivalent to a line-and-sphere type, specifying that the projection includes at least part of a sphere that is at an end of a line that extends, beginning at a control point determined based on the one or more tracked positions of the one or more body parts of the user, outward from the user (Fig. 1C; [0051]: the pointer P is a circle and part of sphere as illustrated in Fig. 1C; the pointer P and a cursor are projected from the device in user’s hand, i.e. a control point, to the center of display as illustrated in Fig. 1C; the direction indicated by the omnidirectional image capturing apparatus 110 and the position of the pointer P coincide; Fig. 14; [0165]: the pointer P is a circle, i.e. a part of a sphere; project the pointer P from a device in a user’s hand to the display as illustrated in Fig. 14).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Faaborg in view of Powderly to include wherein the projection type is equivalent to a line-and-sphere type, specifying that the projection includes at least part of a sphere that is at an end of a line that extends, beginning at a control point determined based on the one or more tracked positions of the one or more body parts of the user, outward from the user as taught by Shohara. The motivation for doing so would have been to indicate a position in the operation screen at which a use operation is input; to obtain an omnidirectional image by synthesizing the two fisheye images based on the eccentric cylinder projection method as taught by Shohara in paragraphs [0054] and [0095].
Regarding to claim 11 (Currently Amended), Faaborg in view of Powderly discloses the non-transitory computer-readable storage medium of claim 10,
the rest claim limitations are similar to claim limitations recited in claim 2. Therefore, same rational used to reject claim 2 is also used to reject claim 11.
Regarding to claim 12 (Currently Amended), Faaborg in view of Powderly discloses the non-transitory computer-readable storage medium of claim 10,
the rest claim limitations are similar to claim limitations recited in claim 3. Therefore, same rational used to reject claim 3 is also used to reject claim 12.
Claims 7-8 and 17-18 are rejected under 35 U.S.C. 103 as being unpatentable over Faaborg (US 20170060230 A1) in view of Powderly (US 20170287225 A1), Ohashi (US 20200218423 A1), and further in view of Hsiao (US 20180335925 A1).
Regarding to claim 7 (Original), Faaborg in view of Powderly discloses the method of claim 1, wherein:
recording intersections of the projection with objects (Faaborg; Fig. 4B; [0034]: a user shifts his gaze toward the scrollable list of objects X, Y and Z, with the user's gaze intersecting a hit area H surrounding the scrollable list of objects X, Y and Z, with the user's gaze coming to rest on object X, as shown in FIG. 4B; Fig. 6A; Fig. 6B; [0037]: the ray emitted by the handheld electronic device intersects the hit area H surrounding the scrollable list of objects X, Y and Z, with the ray coming to rest on and focusing on object X, as shown in FIG. 6B; Fig. 12A-E; [0053]: as the user's eye gaze intersects the user interface, and in particular, the list 20 of objects 20A, as shown in FIG. 12B, the user's focus is established on the list 20 of objects 20A).
Faaborg in view of Powderly fails to explicitly disclose:
a casting direction, for the projection, is along a line connecting one of the user’s eyes to a control point determined based on the one or more tracked positions of the one or more body parts of the user;
the projection: extends away from the user, and falls along the line that intersects the control point and the one of the eyes;
the method further comprises: continuously monitoring, according to movements of the control point and the one of the user's eyes, a path of the projection;
causing a representation of at least part of the path to appear in the artificial reality environment.
In same field of endeavor, Ohashi teaches:
a casting direction, for the projection, is along a line connecting one of the user’s eyes to a control point determined based on the one or more tracked positions of the one or more body parts of the user (Fig. 1; [0045]: the user U10 can select any one of the objects D1 to D4 by means of an instruction using an eye, a fingertip, or the like to perform manipulation corresponding to each object Note; Fig. 1; [0080]: intersection P11 between a straight line connecting the left eye U1 corresponding to the one opened eye and the fingertip U5 and a screen 11);
the projection: extends away from the user, and falls along the line that intersects the control point and the one of the eyes (Fig. 1; [0045]: the user U10 can select any one of the objects D1 to D4 by means of an instruction using an eye, a fingertip, or the like to perform manipulation corresponding to each object Note; Fig. 1; [0080]: intersection P11 between a straight line connecting the left eye U1 corresponding to the one opened eye and the fingertip U5 and a screen 11; [0081]: the left-hand fingertip U5 is used to specify an instruction position, but a right-hand fingertip may be used to specify an instruction position);
causing a representation of at least part of the path to appear in the artificial reality environment (Fig. 13; [0166]: with the display system 2 using virtual reality, a plurality of objects looks double in the virtual space 21; [0172]: the display system 3 is a system using augmented reality (AR) that provides a user interface for achieving intuitive manipulation of the display apparatus 30).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Faaborg in view of Powderly to include a casting direction, for the projection, is along a line connecting one of the user’s eyes to a control point determined based on the one or more tracked positions of the one or more body parts of the user; the projection: extends away from the user, and falls along the line that intersects the control point and the one of the eyes; causing a representation of at least part of the path to appear in the artificial reality environment as taught by Ohashi. The motivation for doing so would have been to select any one of the objects D1 to D4 by means of an instruction using an eye, a fingertip, or the like to perform manipulation corresponding to each object Note; to acquire, by means of imaging, a depth image showing a finger of the user U10 as an image used for detection of the position related to a finger as taught by Ohashi in Fig. 1 and paragraphs [0045] and [0048].
Faaborg in view of Powderly, and Ohashi fails to explicitly disclose:
the method further comprises: continuously monitoring, according to movements of the control point and the one of the user's eyes, a path of the projection;
In same field of endeavor, Hsiao teaches:
the method further comprises: continuously monitoring, according to movements of the control point and the one of the user's eyes, a path of the projection (Fig. 6; [0057]: detect a move 414 of a finger in a space continuously; the move connects the multiple 3D visualizations 405A-C; the user is focusing on the 3D display 10);
causing a representation of at least part of the path to appear in the artificial reality environment (Fig. 6; [0057]: the move connects the multiple 3D visualizations 405A-C; a moving path is displayed as illustrated in Fig. 6; [0058]: represent the 3D visualizations on the 3D display 10; define or create a 3D virtual object 417, and select a range of 3D visualizations).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Faaborg in view of Powderly, and Ohashi to include the method further comprises: continuously monitoring, according to movements of the control point and the one of the user's eyes, a path of the projection; causing a representation of at least part of the path to appear in the artificial reality environment as taught by Hsiao. The motivation for doing so would have been detect a move 414 of a finger in a space; to define or create a 3D virtual object 417, and select a range of 3D visualizations as taught by Hsiao in Fig. 6 and paragraphs [0057-0058].
Regarding to claim 8 (Original), Faaborg in view of Powderly discloses the method of claim 1, wherein:
the projection type is equivalent to a distorted cone type (Powderly; [0108]: cone casting casts a conic or other shape volume 1220 with an adjustable aperture; other shape includes a distorted cone type; [0117]: the cone 1220 may be a cuboid, polyhedron, pyramid, frustum, etc, i.e. distorted cone type; [0159]: the cone may be a geometric cone, a cuboid, a polyhedron, a pyramid, a frustum, or other three-dimensional shapes which may or may not be regular shapes); and
identifying a distorted cone formed with a point of the distorted cone at the one of the user’s eyes and sides of the distorted cone extending through the path of the ray (Powderly; [0108]: cone casting casts a conic or other shape volume 1220 with an adjustable aperture; other shape includes a distorted cone type; Fig. 12A; [0109]: the direction of the cone corresponds to the user's direction of gaze; determine the direction 1224 of the cone using the user's eye pose; [0117]: the cone 1220 has a variety of properties such as, e.g., size, shape, or color).
Faaborg in view of Powderly fails to explicitly disclose:
a casting direction, for the projection, is along a line connecting one of the user’s eyes to a control point determined based on the one or more tracked positions of the one or more body parts of the user; and
the method further comprises:
continuously monitoring, according to movements of the control point and the one of the user’s eyes, a path of a ray that extends away from the user, begins at the control point, and falls along the line that intersects the control point and the user’s dominant eye;
In same field of endeavor, Ohashi teaches:
a casting direction, for the projection, is along a line connecting one of the user’s eyes to a control point determined based on the one or more tracked positions of the one or more body parts of the user (Fig. 1; [0045]: the user U10 can select any one of the objects D1 to D4 by means of an instruction using an eye, a fingertip, or the like to perform manipulation corresponding to each object Note; Fig. 1; [0080]: intersection P11 between a straight line connecting the left eye U1 corresponding to the one opened eye and the fingertip U5 and a screen 11); and
the method further comprises:
a path of a ray that extends away from the user, begins at the control point, and falls along the line that intersects the control point and the user’s dominant eye (Fig. 1; [0045]: the user U10 can select any one of the objects D1 to D4 by means of an instruction using an eye, a fingertip, or the like to perform manipulation corresponding to each object Note; Fig. 1; [0080]: intersection P11 between a straight line connecting the left eye U1 corresponding to the one opened eye and the fingertip U5 and a screen 11; [0081]: he left-hand fingertip U5 is used to specify an instruction position, but a right-hand fingertip may be used to specify an instruction position);
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify Faaborg in view of Powderly to include a casting direction, for the projection, is along a line connecting one of the user’s eyes to a control point determined based on the one or more tracked positions of the one or more body parts of the user; the method further comprises: a path of a ray that extends away from the user, begins at the control point, and falls along the line that intersects the control point and the user’s dominant eye as taught by Ohashi. The motivation for doing so would have been to select any one of the objects D1 to D4 by means of an instruction using an eye, a fingertip, or the like to perform manipulation corresponding to each object Note; to acquire, by means of imaging, a depth image showing a finger of the user U10 as an image used for detection of the position related to a finger as taught by Ohashi in Fig. 1 and paragraphs [0045] and [0048].
Faaborg in view of Powderly, and Ohashi fails to explicitly disclose:
continuously monitoring, according to movements of the control point and the one of the user’s eyes, a path of a ray.
In same field of endeavor, Hsiao teaches:
continuously monitoring, according to movements of the control point and the one of the user’s eyes, a path of a ray (Fig. 6; [0057]: detect a move 414 of a finger in a space conti