DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments
Applicant's arguments filed 02/27/2026 have been fully considered but they are not persuasive. The applicant argues, Spivack fails to teach the amended limitations.
The applicant’s argument is fully considered, but is not persuasive as the amended limitation is taught by Spivack in view of Lehman as detailed in this action.
The applicant argues, Li fails to cure the deficiencies of Spivack.
The applicant’s argument is fully considered, but not persuasive as Lehman teaches the amended limitations as detailed in this action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-3, 5-17, 19-22 is/are rejected under 35 U.S.C. 103 as being unpatentable over Spivack et al. (US 2019/0107990)(Hereinafter referred to as Spivack) in view of Lehman et al. (US 2016/0162020)(Hereinafter referred to as Lehman).
Regarding claim 1, Spivack teaches a computer system configured to communicate with one or more display generation components and one or more input devices (Embodiments of the present disclosure include systems, methods and apparatuses of platforms (e.g., as hosted by the host server 100 as depicted in the example of FIG. 1) of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment. In general, the object or virtual object is generally digitally rendered or synthesized by a machine (e.g., a machine can be one or more of, client device 102 of FIG. 1, client device 402 of FIG. 4A or server 100 of FIG. 1, server 300 of FIG. 3A) to be presented in the AR environment and have human perceptible properties to be human discernible or detectable. See paragraph [0052])( FIG. 4A depicts an example functional block diagram
of a client device 402 such as a mobile device that enables virtual object manipulation and/or virtual object collection for various disclosed applications, in accordance with embodiments of the present disclosure See paragraph [0282]), comprising:
one or more processors (The client device 402 may be any electronic device such as the devices described in conjunction with the client devices 102A-N in the example of FIG. 1 including
but not limited to portable devices, a computer, a server, location-aware devices, mobile phones, PDAs, laptops, palmtops, iPhones, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, smart lens, monocles, smart glasses/goggles, wearable computer such as mobile enabled watches or eyewear, and/or any other mobile interfaces and viewing devices, etc. See paragraph [0283])( One embodiment of the client device 402 further includes a processing unit 434. See paragraph [0292]); and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions (One embodiment of the client device 402 further includes a memory unit 436 and a storage unit 438. The memory unit 436 and a storage unit 438 are, in some embodiments, coupled to the processing unit 434. The memory unit can include volatile and/or non-volatile memory. In rendering or presenting an augmented reality environment, the processing unit 434 can perform one or more processes related to administering an augmented reality workspace in a physical space where a user interface element of the augmented reality workspace is rendered as being present in the physical space and able to be interacted with in the physical space. See paragraph [0294]) for:
displaying, via the one or more display generation components, a first user interface within a three-dimensional environment, wherein the first user interface includes first content corresponding to a first physical item (See figure 2D, 2E)( Through user interfaces of the present disclosure, a user (e.g., User A) for example, can add content, create a new collection of VOBs, add VOBs to a collection, customize the VOBs, edit the VOBs, or further add additional VOBs. The user can also select recipient(s) from their friend list, send any portion of the VOB collection to selected friends (e.g. to user B). This process is also illustrated with further references to the example user interfaces of FIG 5C-FIG. 5G. The user can also have the option to save the VOB collection to a current physical location such that the VOB collection is saved to a location in a public layer associated with the physical location. See paragraph [0193])( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194])( In general, VOBs and/or locations can be addressable with an identifier such as a URL that can be saved, bookmarked, and/or shared. See paragraph [0191]);
while displaying the first user interface, including the first content corresponding to the first physical item, within the three-dimensional environment, receiving, via the one or more input devices, one or more user inputs corresponding to the first content ( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194]); and in
response to receiving the one or more user inputs corresponding to the first content, displaying, via the one or more display generation components, a representation of the first physical item within the three-dimensional environment and outside of the first user interface ( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194])(See figure 2D)( The marketplace of AR environment 230 depicts a VOB 232 having a product listing of a laptop and a VOB 236 showing a product listing of a scooter. See paragraph [0180]), but is silent to wherein the first user interface is an environment-locked virtual object anchored to a location within the three-dimensional environment.
Lehman teaches an augmented reality system in which applications can be launched on the fly and world locked to specific surfaces such that they stay in the same place even as the user’s head moves (In augmented reality environments, the application launcher allows a user to launch applications “out of thin air” or as an overlay on any suitable surface in the real-world environment without leaving the augmented reality experience. In such environments, the application launcher may be world-locked, such that the launcher appears to stay in the same position relative to real-world objects, regardless of the movements of the head-mounted display device See paragraph [0012]).
Spivack and Lehman teach of augmented reality systems for displaying virtual objects and interfaces and Lehman teaches that the virtual interface can be world-locked to specific surfaces, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Spivack with the world-locked virtual interface technique of Lehman such that the system could lock virtual user-interface elements to specific surfaces they are associated with.
Regarding claim 2, Spivack in view of Lehman teaches The computer system of claim 1, wherein the three-dimensional environment is an optical passthrough environment (Spivack; The client device 402 may be any electronic
device such as the devices described in conjunction with the client devices 102A-N in the example of FIG. 1 including but not limited to portable devices, a computer, a server, location-aware devices, mobile phones, PDAs, laptops, palmtops, iPhones, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, smart lens, monocles, smart glasses/goggles, wearable computer such as mobile enabled watches or eyewear, and/or any other mobile interfaces and viewing devices, etc. See paragraph [0282]).
Regarding claim 3, Spivack in view of Lehman teaches The computer system of claim 1, wherein the three-dimensional environment is a virtual passthrough environment (Spivack; The client device 402 may be any electronic
device such as the devices described in conjunction with the client devices 102A-N in the example of FIG. 1 including but not limited to portable devices, a computer, a server, location-aware devices, mobile phones, PDAs, laptops, palmtops, iPhones, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, smart lens, monocles, smart glasses/goggles, wearable computer such as mobile enabled watches or eyewear, and/or any other mobile interfaces and viewing devices, etc. See paragraph [0282]).
Regarding claim 5, Spivack in view of Lehman teaches The computer system of claim 1, wherein the one or more user inputs corresponding to the first content comprises one or more air gesture inputs corresponding to the first content (Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; For instance, users can aim the selector and pinch
to zoom using hand gestures in front of a camera and also on a touch screen input method. This allows the user to shuffle through virtual objects to select the object they would like to target and select. This could be in the vein of a missile tracking system. See paragraph [0310]) (Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140]).
Regarding claim 6, Spivack in view of Lehman teaches The computer system of claim 5, wherein the one or more air gesture inputs corresponding to the first content comprises a pinch air gesture corresponding to the first content (Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; For instance, users can aim the selector and pinch
to zoom using hand gestures in front of a camera and also on a touch screen input method. This allows the user to shuffle through virtual objects to select the object they would like to target and select. This could be in the vein of a missile tracking system. See paragraph [0310]) (Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140]).
Regarding claim 7, Spivack in view of Lehman teaches The computer system of claim 6, wherein the one or more air gesture inputs comprise movement of the pinch air gesture in a first direction (Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; For instance, users can aim the selector and pinch
to zoom using hand gestures in front of a camera and also on a touch screen input method. This allows the user to shuffle through virtual objects to select the object they would like to target and select. This could be in the vein of a missile tracking system. See paragraph [0310]) ( Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140]).
Regarding claim 8, Spivack in view of Lehman teaches The computer system of claim 1, wherein the representation of the first physical item is displayed at a first size that is determined based on one or more physical dimensions of the first physical item (Spivack; Moreover, a model of the product offering, can be rendered in the virtual object at the physical location, in process 1306. For example, a real life size of the furniture listing available for sale or rent can be made to appear in a unit in the building for visualization. The apartment buyer or renter can manipulate the VOB of the furniture listing to rearrange its location or position, change the color, pattern, change the finish (e.g., fabric, leather, wood, stainless steel, chrome, etc.) to visualize how it fits into the space and to examine or select colors, patterns, etc. See paragraph [0458]).
Regarding claim 9, Spivack in view of Lehman teaches The computer system of claim 1, the one or more programs further including instructions for: while displaying the representation of the first physical item within the three-dimensional environment and outside of the first user interface, receiving, via the one or more input devices, a first set of user inputs corresponding to the representation of the first physical item; and in response to receiving the first set of user inputs, displaying, via the one or more display generation components, movement of the representation of the first physical item within the three-dimensional environment (Spivack; Similarly, VOB 236 can include a model or 3D model of an actual scooter. The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184])( Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140]).
Regarding claim 10, Spivack in view of Lehman teaches the computer system of claim 1, wherein the first user interface is a messaging user interface (Spivack; Embodiments of the present disclosure include systems, methods and apparatuses of platforms (e.g., as hosted by the host server 100 as depicted in the example of FIG. 1) of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment. In general, the object or virtual object is generally digitally rendered or synthesized by a machine (e.g., a machine can be one or more of, client device 102 of FIG. 1, client device 402 of FIG. 4A or server 100 of FIG. 1, server 300 of FIG. 3A) to be presented in the AR environment and have human perceptible properties to be human discernible or detectable. See paragraph [0052]).
Regarding claim 11, Spivack in view of Lehman teaches The computer system of claim 10, wherein the first content corresponding to the first physical item comprises a first message transmitted to a user of the computer system by another user (Spivack; If the VOB collection is shared with or sent to User
B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. B can have the option to reply to User A and/or to add content to the VOB collection of User A. See paragraph [0194]).
Regarding claim 12, Spivack in view of Lehman teaches The computer system of claim 1, wherein the first content corresponding to the first physical item comprises a uniform resource locator corresponding to the first physical item ( Spivack; In general, VOBs and/or locations can be addressable with an identifier such as a URL that can be saved, bookmarked, and/or shared. See paragraph [0191]).
Regarding claim 13, Spivack in view of Lehman teaches The computer system of claim 1, wherein displaying the representation of the first physical item within the three-dimensional environment and outside of the first user interface further comprises: displaying, via the one or more display generation components, the representation of the first physical item at a first position in the three-dimensional environment while concurrently displaying the first user interface at a second position in the three-dimensional environment different from the first position (Spivack; Embodiments of the present disclosure also include group messages as virtual objects, or a mailing list as a virtual object. User Joe can post a message or a message in a VOB for another user or multiple users. Note that when a user (Joe) posts a VOB for another user (Kathy), the VOB can be world locked, or it can be wherever the recipient user (Kathy) is (user locked). Users and groups can share objects as messages or lock them to locations that only members of a group can see, for example, VOB 212 can be locked to the coffee table location or anywhere in the home of Joe and Kathy such that only their family can see the message contents. In one embodiment, when a VOB is shared with another user, it can default to a relative position and/or locked position. For example, relative locking of a VOB to a recipient user's inbox location can be an example of a default setting. See paragraph [0176]).
Regarding claim 14, Spivack in view of Lehman teaches The computer system of claim 13, the one or more programs further including instructions for: while concurrently displaying the representation of the first physical item at the first position within the three-dimensional environment and the first user interface at the second position in the three-dimensional environment, receiving, via the one or more input devices, a second set of user inputs corresponding to the first user interface; and in response to receiving the second set of user inputs, displaying, via the one or more display generation components, movement of the first user interface within the three-dimensional environment from the second position to a third position in the three-dimensional environment different from the first position and the second position, while maintaining display of the representation of the first physical item at the first position in the three-dimensional environment (Spivack; Embodiments of the present disclosure also include group messages as virtual objects, or a mailing list as a virtual object. User Joe can post a message or a message in a VOB for another user or multiple users. Note that when a user (Joe) posts a VOB for another user (Kathy), the VOB can be world locked, or it can be wherever the recipient user (Kathy) is (user locked). Users and groups can share objects as messages or lock them to locations that only members of a group can see, for example, VOB 212 can be locked to the coffee table location or anywhere in the home of Joe and Kathy such that only their family can see the message contents. In one embodiment, when a VOB is shared with another user, it can default to a relative position and/or locked position. For example, relative locking of a VOB to a recipient user's inbox location can be an example of a default setting. See paragraph [0176]) (Spivack; Similarly, VOB 236 can include a model or 3D model of an actual scooter. The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184])( Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140]).
Regarding claim 15, Spivack in view of Lehman teaches The computer system of claim 13, the one or more programs further including instructions for: while concurrently displaying the representation of the first physical item at the first position within the three-dimensional environment and the first user interface at the second position in the three-dimensional environment, receiving, via the one or more input devices, a third set of user inputs corresponding to the representation of the first physical item; and in response to receiving the third set of user inputs, displaying, via the one or more display generation components, movement of the representation of the first physical item within the three-dimensional environment from the first position to a fourth position in the three-dimensional environment different from the first position and the second position, while maintaining display of the first user interface at the second position in the three-dimensional environment (Spivack; Embodiments of the present disclosure also include group messages as virtual objects, or a mailing list as a virtual object. User Joe can post a message or a message in a VOB for another user or multiple users. Note that when a user (Joe) posts a VOB for another user (Kathy), the VOB can be world locked, or it can be wherever the recipient user (Kathy) is (user locked). Users and groups can share objects as messages or lock them to locations that only members of a group can see, for example, VOB 212 can be locked to the coffee table location or anywhere in the home of Joe and Kathy such that only their family can see the message contents. In one embodiment, when a VOB is shared with another user, it can default to a relative position and/or locked position. For example, relative locking of a VOB to a recipient user's inbox location can be an example of a default setting. See paragraph [0176]) (Spivack; Similarly, VOB 236 can include a model or 3D model of an actual scooter. The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184])( Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140]).
Regarding claim 16, Spivack in view of Lehman teaches The computer system of claim 1, the one or more programs further including instructions for: while displaying the representation of the first physical item within the three-dimensional environment and outside of the first user interface, receiving, via the one or more input devices, a fourth set of user inputs corresponding to the representation of the first physical item; in response to receiving the fourth set of user inputs, displaying, via the one or more display generation components, a first set of options corresponding to the first physical item; while displaying the first set of options corresponding to the first physical item, receiving, via the one or more input devices, a fifth set of user inputs corresponding to a first option of the first set of options; and in response to receiving the fifth set of user inputs, modifying one or more visual characteristics of the representation of the first physical item based on the first option (Spivack; Embodiments of the present disclosure also include group messages as virtual objects, or a mailing list as a virtual object. User Joe can post a message or a message in a VOB for another user or multiple users. Note that when a user (Joe) posts a VOB for another user (Kathy), the VOB can be world locked, or it can be wherever the recipient user (Kathy) is (user locked). Users and groups can share objects as messages or lock them to locations that only members of a group can see, for example, VOB 212 can be locked to the coffee table location or anywhere in the home of Joe and Kathy such that only their family can see the message contents. In one embodiment, when a VOB is shared with another user, it can default to a relative position and/or locked position. For example, relative locking of a VOB to a recipient user's inbox location can be an example of a default setting. See paragraph [0176]) (Spivack; Similarly, VOB 236 can include a model or 3D model of an actual scooter. The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184])( Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140])( Spivack; Plurality of interaction options for movement, and properties of the physical item).
Regarding claim 17, Spivack in view of Lehman teaches The computer system of claim 1, the one or more programs further including instructions for: while displaying the representation of the first physical item within the three-dimensional environment and outside of the first user interface, receiving, via the one or more input devices, a sixth set of user inputs corresponding to the representation of the first physical item; and in response to receiving the sixth set of user inputs, displaying, via the one or more display generation components, a purchase option that is selectable to initiate a process for ordering the first physical item (Spivack; Embodiments of the present disclosure also include group messages as virtual objects, or a mailing list as a virtual object. User Joe can post a message or a message in a VOB for another user or multiple users. Note that when a user (Joe) posts a VOB for another user (Kathy), the VOB can be world locked, or it can be wherever the recipient user (Kathy) is (user locked). Users and groups can share objects as messages or lock them to locations that only members of a group can see, for example, VOB 212 can be locked to the coffee table location or anywhere in the home of Joe and Kathy such that only their family can see the message contents. In one embodiment, when a VOB is shared with another user, it can default to a relative position and/or locked position. For example, relative locking of a VOB to a recipient user's inbox location can be an example of a default setting. See paragraph [0176]) (Spivack; Similarly, VOB 236 can include a model or 3D model of an actual scooter. The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184])( Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140])( Spivack; Plurality of interaction options for movement, and properties of the physical item) (Spivack; Furthermore, any of the above interaction types can be used to initiate a transaction with products associated with the VOBs 232 or 236. For example a pedestrian passing by the VOB 232 can stop to browse it, test it, interact with the model of it and decide to purchase it. The pedestrian can initiate the purchase through various features of the VOB 232, pay for the laptop represented by the VOB 232 and set a delivery date and provide delivery instructions where applicable. See paragraph [0186]).
Regarding claim 19, Spivack in view of Lehman teaches The computer system of claim 1, the one or more programs further including instructions for: while displaying the representation of the first physical item within the three-dimensional environment and outside of the first user interface, receiving, via the one or more input devices, an eighth set of user inputs corresponding to the representation of the first physical item; and in response to receiving the eighth set of user inputs, displaying, via the one or more display generation components, movement of a first portion of the representation of the first physical item without moving a second portion of the representation of the first physical item different from the first portion (Spivack; The VOB 232 can include a model or 3D model of an actual laptop and the user can interact with it, to see it from various aspects, see it in laptop mode vs. note book mode, test the keys, etc. The user may also be able to see the laptop in different color finishes ( e.g, black or silver or chrome, etc.) and the user may be able to test out any customizable features prior to purchase (e.g., screen resolution, custom color, CPU speed, memory size, etc.), test out various software options etc. See paragraph [0183]).
Regarding claim 20, Spivack in view of Lehman teaches The computer system of claim 19, wherein displaying movement of the first portion of the representation of the first physical item comprises moving the first portion of the representation of the first physical item from a stowed configuration to an extended configuration (Spivack; The VOB 232 can include a model or 3D model of an actual laptop and the user can interact with it, to see it from various aspects, see it in laptop mode vs. note book mode, test the keys, etc. The user may also be able to see the laptop in different color finishes ( e.g, black or silver or chrome, etc.) and the user may be able to test out any customizable features prior to purchase (e.g., screen resolution, custom color, CPU speed, memory size, etc.), test out various software options etc. See paragraph [0183]).
Regarding claim 21, Spivack teaches A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a computer system that is in communication with one or more display generation components and one or more input devices, the one or more programs including instructions (The client device 402 may be any electronic device such as the devices described in conjunction with the client devices 102A-N in the example of FIG. 1 including but not limited to portable devices, a computer, a server, location-aware devices, mobile phones, PDAs, laptops, palmtops, iPhones, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, smart lens, monocles, smart glasses/goggles, wearable computer such as mobile enabled watches or eyewear, and/or any other mobile interfaces and viewing devices, etc. See paragraph [0283])( One embodiment of the client device 402 further includes a processing unit 434. See paragraph [0292]) (One embodiment of the client device 402 further includes a memory unit 436 and a storage unit 438. The memory unit 436 and a storage unit 438 are, in some embodiments, coupled to the processing unit 434. The memory unit can include volatile and/or non-volatile memory. In rendering or presenting an augmented reality environment, the processing unit 434 can perform one or more processes related to administering an augmented reality workspace in a physical space where a user interface element of the augmented reality workspace is rendered as being present in the physical space and able to be interacted with in the physical space. See paragraph [0294]) for:
displaying, via the one or more display generation components, a first user interface within a three-dimensional environment, wherein the first user interface includes first content corresponding to a first physical item (See figure 2D, 2E)( Through user interfaces of the present disclosure, a user (e.g., User A) for example, can add content, create a new collection of VOBs, add VOBs to a collection, customize the VOBs, edit the VOBs, or further add additional VOBs. The user can also select recipient(s) from their friend list, send any portion of the VOB collection to selected friends (e.g. to user B). This process is also illustrated with further references to the example user interfaces of FIG 5C-FIG. 5G. The user can also have the option to save the VOB collection to a current physical location such that the VOB collection is saved to a location in a public layer associated with the physical location. See paragraph [0193])( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194])( In general, VOBs and/or locations can be addressable with an identifier such as a URL that can be saved, bookmarked, and/or shared. See paragraph [0191]);
while displaying the first user interface, including the first content corresponding to the first physical item, within the three-dimensional environment, receiving, via the one or more input devices, one or more user inputs corresponding to the first content ( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194]); and
in response to receiving the one or more user inputs corresponding to the first content, displaying, via the one or more display generation components, a representation of the first physical item within the three-dimensional environment and outside of the first user interface ( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194])(See figure 2D)( The marketplace of AR environment 230 depicts a VOB 232 having a product listing of a laptop and a VOB 236 showing a product listing of a scooter. See paragraph [0180]), but is silent to wherein the first user interface is an environment-locked virtual object anchored to a location within the three-dimensional environment.
Lehman teaches an augmented reality system in which applications can be launched on the fly and world locked to specific surfaces such that they stay in the same place even as the user’s head moves (In augmented reality environments, the application launcher allows a user to launch applications “out of thin air” or as an overlay on any suitable surface in the real-world environment without leaving the augmented reality experience. In such environments, the application launcher may be world-locked, such that the launcher appears to stay in the same position relative to real-world objects, regardless of the movements of the head-mounted display device See paragraph [0012]).
Spivack and Lehman teach of augmented reality systems for displaying virtual objects and interfaces and Lehman teaches that the virtual interface can be world-locked to specific surfaces, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Spivack with the world-locked virtual interface technique of Lehman such that the system could lock virtual user-interface elements to specific surfaces they are associated with.
Regarding claim 22, Spivack teaches a method (Embodiments of the present disclosure include systems, methods and apparatuses of platforms (e.g., as hosted by the host server 100 as depicted in the example of FIG. 1) of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment. In general, the object or virtual object is generally digitally rendered or synthesized by a machine (e.g., a machine can be one or more of, client device 102 of FIG. 1, client device 402 of FIG. 4A or server 100 of FIG. 1, server 300 of FIG. 3A) to be presented in the AR environment and have human perceptible properties to be human discernible or detectable. See paragraph [0052])( FIG. 4A depicts an example functional block diagram
of a client device 402 such as a mobile device that enables virtual object manipulation and/or virtual object collection for various disclosed applications, in accordance with embodiments of the present disclosure See paragraph [0282]), comprising:
at a computer system that is in communication with one or more display generation components and one or more input devices (Embodiments of the present disclosure include systems, methods and apparatuses of platforms (e.g., as hosted by the host server 100 as depicted in the example of FIG. 1) of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment. In general, the object or virtual object is generally digitally rendered or synthesized by a machine (e.g., a machine can be one or more of, client device 102 of FIG. 1, client device 402 of FIG. 4A or server 100 of FIG. 1, server 300 of FIG. 3A) to be presented in the AR environment and have human perceptible properties to be human discernible or detectable. See paragraph [0052])( FIG. 4A depicts an example functional block diagram of a client device 402 such as a mobile device that enables virtual object manipulation and/or virtual object collection for various disclosed applications, in accordance with embodiments of the present disclosure See paragraph [0282]):
displaying, via the one or more display generation components, a first user interface within a three-dimensional environment, wherein the first user interface includes first content corresponding to a first physical item (See figure 2D, 2E)( Through user interfaces of the present disclosure, a user (e.g., User A) for example, can add content, create a new collection of VOBs, add VOBs to a collection, customize the VOBs, edit the VOBs, or further add additional VOBs. The user can also select recipient(s) from their friend list, send any portion of the VOB collection to selected friends (e.g. to user B). This process is also illustrated with further references to the example user interfaces of FIG 5C-FIG. 5G. The user can also have the option to save the VOB collection to a current physical location such that the VOB collection is saved to a location in a public layer associated with the physical location. See paragraph [0193])( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194])( In general, VOBs and/or locations can be addressable with an identifier such as a URL that can be saved, bookmarked, and/or shared. See paragraph [0191]);
while displaying the first user interface, including the first content corresponding to the first physical item, within the three-dimensional environment, receiving, via the one or more input devices, one or more user inputs corresponding to the first content ( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194]); and
in response to receiving the one or more user inputs corresponding to the first content, displaying, via the one or more display generation components, a representation of the first physical item within the three-dimensional environment and outside of the first user interface ( If the VOB collection is shared with or sent to User B, User B can receive a notification in their inbox. User B can open the VOB collection sent by the User A. The VOB collection created by user A can appear in front of User B. User B can have the option to reply to User A and/or to add content to the VOB collection of User A. If not, User B can create their own collection of VOBs. If the User B adds content to User A's VOB collection, the User B can also customize the VOBs, edit the VOBs, and/or add more VOBs. Once completed, the User B can send a rely to User A, and the User A can receive a notification in their inbox. This process is also illustrated with further references to the example user interfaces of FIG. 5C-FIG. 5G. See paragraph [0194])(See figure 2D)( The marketplace of AR environment 230 depicts a VOB 232 having a product listing of a laptop and a VOB 236 showing a product listing of a scooter. See paragraph [0180]), but is silent to wherein the first user interface is an environment-locked virtual object anchored to a location within the three-dimensional environment.
Lehman teaches an augmented reality system in which applications can be launched on the fly and world locked to specific surfaces such that they stay in the same place even as the user’s head moves (In augmented reality environments, the application launcher allows a user to launch applications “out of thin air” or as an overlay on any suitable surface in the real-world environment without leaving the augmented reality experience. In such environments, the application launcher may be world-locked, such that the launcher appears to stay in the same position relative to real-world objects, regardless of the movements of the head-mounted display device See paragraph [0012]).
Spivack and Lehman teach of augmented reality systems for displaying virtual objects and interfaces and Lehman teaches that the virtual interface can be world-locked to specific surfaces, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Spivack with the world-locked virtual interface technique of Lehman such that the system could lock virtual user-interface elements to specific surfaces they are associated with.
Claim(s) 18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Spivack et al. (US 2019/0107990)(Hereinafter referred to as Spivack) in view of Lehman et al. (US 2016/0162020)(Hereinafter referred to as Lehman) in view of Li et al. (US 2020/0258315)(Hereinafter referred to as Li).
Regarding claim 18, Spivack in view of Lehman teaches the computer system of claim 1, the one or more programs further including instructions for: while displaying the representation of the first physical item within the three-dimensional environment and outside of the first user interface, receiving, via the one or more input devices, a seventh set of user inputs corresponding to the representation of the first physical item (Spivack; Embodiments of the present disclosure also include group messages as virtual objects, or a mailing list as a virtual object. User Joe can post a message or a message in a VOB for another user or multiple users. Note that when a user (Joe) posts a VOB for another user (Kathy), the VOB can be world locked, or it can be wherever the recipient user (Kathy) is (user locked). Users and groups can share objects as messages or lock them to locations that only members of a group can see, for example, VOB 212 can be locked to the coffee table location or anywhere in the home of Joe and Kathy such that only their family can see the message contents. In one embodiment, when a VOB is shared with another user, it can default to a relative position and/or locked position. For example, relative locking of a VOB to a recipient user's inbox location can be an example of a default setting. See paragraph [0176]) (Spivack;Similarly, VOB 236 can include a model or 3D model of an actual scooter. The user can interact with the VOB 236 by moving it around, changing the color/finish, testing/viewing different available sizes, etc., prior to making a decision on whether to purchase one. See paragraph [0184])( Spivack; The interaction with the VOB can be initiated by voice, touch, eye, gaze, gesture (body, hand, head, arms, legs, limbs, eyes, torso, etc.), text input and/or other command submitted by a user with respect to the object. See paragraph [0185])( Spivack; The input mechanism on client devices 102A-N can include touch screen keypad (including single touch, multi-touch, gesture sensing in 2D or 3D, etc.), a physical keypad, a mouse, a pointer, a track pad, motion detector ( e.g., including I-axis, 2-axis, 3-axis accelerometer, etc.), a light sensor, capacitance sensor, resistance sensor, temperature sensor, proximity sensor, a piezoelectric device, device orientation detector ( e.g., electronic compass, tilt sensor, rotation sensor, gyroscope, accelerometer), eye tracking, eye detection, pupil tracking/detection, or a combination of the above. See paragraph [0140])( Spivack; Plurality of interaction options for movement, and properties of the physical item) (Spivack; Furthermore, any of the above interaction types can be used to initiate a transaction with products associated with the VOBs 232 or 236. For example a pedestrian passing by the VOB 232 can stop to browse it, test it, interact with the model of it and decide to purchase it. The pedestrian can initiate the purchase through various features of the VOB 232, pay for the laptop represented by the VOB 232 and set a delivery date and provide delivery instructions where applicable. See paragraph [0186]), but is silent to and in response to receiving the seventh set of user inputs, displaying, via the one or more display generation components, physical dimension information corresponding to the first physical item.
Li teaches an augmented reality system in which a virtual ruler can be placed in the scene to represent dimensions relative to a virtual object (If a mating button is generated, a corresponding offset ruler 640 is also generated and displayed along the corresponding ray, displaying a distance from the virtual object to the RW environment surface, as shown by block 242 (FIG. 2C), displayed as evenly spaced hash lines along the ruler 640. A start position of the ruler 640 is the center of the bounding box 335 for the virtual object 330. An end position of the ruler 640 is the center of the corresponding mating button 645. The unit distance spacing the hash lines on the ruler 640 in this example embodiment is 0.1 m, although different units may be either selected or defaulted according to the size of the virtual object 330 and/or the size of the VR environment. When the user gazes at a point of the ruler 640, a slider 650 (shown in green in FIG. 6B) appears at the position on the ruler gazed upon by the user. See paragraph [0079]).
Spivack in view of Lehman and Li teach of presenting virtual objects in augmented reality environments and Li teaches that dimensions can be placed that indicated distances corresponding to the first physical item, therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the invention to combine the system of Spivick in view of Lehman with the virtual ruler techniques of Li such that the user could visualize dimensions relative to a physical object.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to NICHOLAS R WILSON whose telephone number is (571)272-0936. The examiner can normally be reached M-F 7:30-5:00PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kee Tung can be reached at (572)-272-7794. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/NICHOLAS R WILSON/Primary Examiner, Art Unit 2611
Prosecution Insights