Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Applicant’s Arguments
Applicant’s arguments filed 03/25/2026 have been fully considered, but they are not deemed to be persuasive.
Applicant argues that the cited references do not teach "identifying... at least one physical characteristic associated with the object" to display content that "is anchored to the object at a location based on the at least one characteristic" which is not persuasive. Tichenor teaches the claimed " an identification, in the image, of an object associated with the command and at least one physical characteristic associated with the object identified from the image" ([0111] - At block 804, process 800 can determine properties of the surface and/or an associated context. Surfaces can have properties such as a type of objects that can be placed on the surface, a surface shape, a location in the artificial reality environment, a list of augments on the surface, meta-tags (e.g., machine learning tagging such as for recognition of real-world objects on surface, surface type, etc.), a layout, or other characteristics); and “the action including a display of content associated with the command on the device such that the content is anchored to the object at a location based on the at least one physical characteristic" ([0111] - A surface layout can be user selected or automatically selected e.g., based on surface size; surface shape; number, size, or type of augments placed on the surface, etc. In some implementations, a layout can be dynamic e.g., a first item added to the surface goes in the middle, a second added augment moves first so the layout is two side-by-side elements, a third added augment moves first and second, so the layout is three equally spaced side-by-side elements, etc).
Furthermore, Applicant argues that “Tichenor does not identify a physical characteristic of an object in an image to anchor content” which is not persuasive because as showed above (e.g., paragraph [0111]), Tichenor teaches the claimed “identify a physical characteristic of an object in an image to anchor content” (Tichenor, [0107] - Surfaces can automatically be assigned some properties such as size, orientation, shape, or meta-tags (based on object recognition), and users can define other properties which may be from a pre-defined set of property categories). See also Tichenor’s physical characteristics of the surface comprises “size, orientation, shape, or meta-tags (based on object recognition), and users can define other properties which may be from a pre-defined set of property categories” (e.g., [0107]) which teaches the claimed “the at least one physical characteristic includes a depth, a distance, a direction, or a size of the object to support the command” of claim 7.
Accordingly, the claimed invention as represented in the claims does not represent a patentable distinction over the art of record.
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over TICHENOR et al (US 20220068035) in view of Pustejovsky et al (Embodied Human- Computer Interactions through Situated Grounding).
As per claim 1, Tichenor teaches the claimed "method" comprising: "identifying a command from a user of a device" (Tichenor, [0112] - an augment can be associated with a surface in response to a user placing it on the surface (or otherwise performing an interaction to connect the augment to the surface, e.g., based on user attention, a voice command, etc.)); "in response to the command, identifying an image associated with a gaze of the user" (Tichenor, [0125] - The user can further move the augment, e.g., by indicating a new surface with the users gaze, by performing a gesture (e.g., "dragging" the augment to a new location), with a voice command, etc.); "identifying an action to the command and the image" (Tichenor, [0123] - In response to a user command (e.g., activating a UI element-not shown) the artificial reality system attached a virtual tablet 1204 in the hand 1202 of the user, allowing the user to make various selections. In this example, the user has selected a pictures option 1206 from the tablet 1204 and selected (not shown) a picture to add to the artificial reality environment; [0125] - The augment 1302 has multiple display modes. A first display mode was enabled due to the augment 1302 being on vertical surface 1102, causing the augment to be shaped as a wall hanging. A second display mode has an alternate condition that is true when the augment 1302 is on horizontal surface, such as surface 1108. By enabling this second display mode in response to being placed on horizontal surface 1108, augment 1302 reconfigures itself to appear as a standing picture frame; [0027] - An augment can be created by requesting the augment from the artificial reality system shell, where the request supplies a manifest specifying initial properties of the augment. The manifest can specify parameters such as an augment title, a type for the augment, display properties (size, orientation, location, eligible location type, etc.) for the augment in different display modes or contexts, context factors the augment needs to be informed of to enable display modes or invoke logic, etc); “the application of the language model including an identification, in the image, of an object for associated with the command and at least one physical characteristic associated with the object identified from the image” (Ticheno, [0111] - At block 804, process 800 can determine properties of the surface and/or an associated context. Surfaces can have properties such as a type of objects that can be placed on the surface, a surface shape, a location in the artificial reality environment, a list of augments on the surface, meta-tags (e.g., machine learning tagging such as for recognition of real-world objects on surface, surface type, etc.), a layout, or other characteristics); and “initiating the action in association with the object, the action including a display of content associated with the command on the device such that the content is anchored to the object at a location based on the at least one physical characteristic" ([0111] - A surface layout can be user selected or automatically selected e.g., based on surface size; surface shape; number, size, or type of augments placed on the surface, etc. In some implementations, a layout can be dynamic e.g., a first item added to the surface goes in the middle, a second added augment moves first so the layout is two side-by-side elements, a third added augment moves first and second, so the layout is three equally spaced side-by-side elements, etc). It is noted that Tichenor does not explicitly teach “the application of the language model”; however, Ticheno’s voice command (e.g., [0125] - The user can further move the augment, e.g., by indicating a new surface with the users gaze, by performing a gesture (e.g., "dragging" the augment to a new location), with a voice command, etc.) suggests the application of a language model as claimed; see also Pustejovsky, Abstract - a simulation platform for modeling and building Embodied Human-Computer Interactions (EHCI). This system, VoxWorld, is a multimodal dialogue system enabling communication through language, gesture, action, facial expressions, and gaze tracking, in the context of task-oriented interactions. A multimodal simulation is an embodied 3D virtual realization of both the situational environment and the co-situated agents, as well as the most salient content denoted by communicative acts in a discourse. Thus, it would have been obvious, in view of Pustejovsky, to configure Tichenor's method by providing an application of a language model to define an action based on the user's command and gaze direction. The motivation is to provide a venue for the human and computer to share an epistemic space, and any communicative modality that can be expressed within that space (e.g., linguistic, visual, gestural) enriches the number of ways that a human and a computer can communicate regarding objects, actions, and situation-based tasks (Pustejovsky, 4 Conclusion).
Claim 2 adds into claim 1 " wherein identifying the action based on the application of the language model to the command and the image comprises identifying the content for the display and an orientation for the content based on at least one physical characteristic, and wherein initiating the action comprises causing display of the content in the orientation on the a display" (Tichenor, [0107] - Surfaces can automatically be assigned some properties such as size, orientation, shape, or meta-tags (based on object recognition), and users can define other properties which may be from a pre-defined set of property categories; [0108] - When an augment is placed on a surface, properties of the surface can be provided to the augment, which the augment can use to configure its display or actions; [0125] - The augment 1302 has multiple display modes. A first display mode was enabled due to the augment 1302 being on vertical surface 1102, causing the augment to be shaped as a wall hanging. A second display mode has an alternate condition that is true when the augment 1302 is on horizontal surface, such as surface 1108. By enabling this second display mode in response to being placed on horizontal surface 1108, augment 1302 reconfigures itself to appear as a standing picture frame. When the augment 1302 is placed on the surface 1108, the surface 1108 can identify a slot for the augment. In this example, surface 1108 has a layout with slots 1346, 1348, and 1350. There is already an object 1012 in slot 1350 and slot 1348 is too small for the augment 1302 in the current picture frame shape, thus surface 1108 selects slot 1346 for the augment 1302 and informs augment 1302 of its location on the surface 1108, allowing augment 1302 to set its location within the artificial reality environment to be in slot 1346; [0027] - An augment can be created by requesting the augment from the artificial reality system shell, where the request supplies a manifest specifying initial properties of the augment. The manifest can specify parameters such as an augment title, a type for the augment, display properties (size, orientation, location, eligible location type, etc.) for the augment in different display modes or contexts, context factors the augment needs to be informed of to enable display modes or invoke logic, etc.) (Noted: Ticheno’s displaying of the augment depends on the physical property (e.g., vertical, horizontal) of the surface in which the augment’s orientation changes according to the vertical surface (e.g., as a wall hanging parallel to the vertical surface) or the horizontal surface (e.g., as a standing picture frame with its orientation associated with its assigned slot)).
Claim 3 adds into claim 2 " wherein identifying the orientation for the content on the display includes identifying a size for the content on the display " (Tichenor, [0027] - An augment can be created by requesting the augment from the artificial reality system shell, where the request supplies a manifest specifying initial properties of the augment. The manifest can specify parameters such as an augment title, a type for the augment, display properties (size, orientation, location, eligible location type, etc.) for the augment in different display modes or contexts, context factors the augment needs to be informed of to enable display modes or invoke logic, etc; [0125] - By enabling this second display mode in response to being placed on horizontal surface 1108, augment 1302 reconfigures itself to appear as a standing picture frame. When the augment 1302 is placed on the surface 1108, the surface 1108 can identify a slot for the augment. In this example, surface 1108 has a layout with slots 1346, 1348, and 1350. There is already an object 1012 in slot 1350 and slot 1348 is too small for the augment 1302 in the current picture frame shape, thus surface 1108 selects slot 1346 for the augment 1302 and informs augment 1302 of its location on the surface 1108, allowing augment 1302 to set its location within the artificial reality environment to be in slot 1346) (Noted: Ticheno’s displaying of the augment depends on the physical property (e.g., vertical, horizontal) of the surface and the augmented size in which the augment’s orientation changes according to the vertical surface (e.g., as a wall hanging parallel to the vertical surface) or the horizontal surface (e.g., as a standing picture frame with its orientation associated with its assigned slot to fit the augment’s size)).
Claim 4 adds into claim 2 "wherein the action overlays the content on the object relative to the gaze of the user" (Tichenor, [0125] - The user can further move the augment, e.g., by indicating a new surface with the users gaze, by performing a gesture (e.g., "dragging" the augment to a new location), with a voice command, etc.).
Claim 5 adds into claim 2 "wherein identifying the orientation for the content on the display of the device comprises: identifying a depth, a distance, a direction, or a size of the object in the image” (Tichenor, [0107] - Surfaces can automatically be assigned some properties such as size, orientation, shape, or meta-tags (based on object recognition), and users can define other properties which may be from a pre-defined set of property categories; [0108] - When an augment is placed on a surface, properties of the surface can be provided to the augment, which the augment can use to configure its display or actions); and “identifying the orientation based on the depth, the distance, the direction, or the size of the object in the image" (Tichenor, [0027] - An augment can be created by requesting the augment from the artificial reality system shell, where the request supplies a manifest specifying initial properties of the augment. The manifest can specify parameters such as an augment title, a type for the augment, display properties (size, orientation, location, eligible location type, etc.) for the augment in different display modes or contexts, context factors the augment needs to be informed of to enable display modes or invoke logic, etc; [0125] - By enabling this second display mode in response to being placed on horizontal surface 1108, augment 1302 reconfigures itself to appear as a standing picture frame. When the augment 1302 is placed on the surface 1108, the surface 1108 can identify a slot for the augment. In this example, surface 1108 has a layout with slots 1346, 1348, and 1350. There is already an object 1012 in slot 1350 and slot 1348 is too small for the augment 1302 in the current picture frame shape, thus surface 1108 selects slot 1346 for the augment 1302 and informs augment 1302 of its location on the surface 1108, allowing augment 1302 to set its location within the artificial reality environment to be in slot 1346) (Noted: Ticheno’s displaying of the augment depends on the physical property (e.g., vertical, horizontal) of the surface and the augmented size in which the augment’s orientation changes according to the vertical surface (e.g., as a wall hanging parallel to the vertical surface) or the horizontal surface (e.g., as a standing picture frame with its orientation associated with its assigned slot to fit the augment’s size)).
Claim 6 adds into claim 1 "wherein the action includes at least one application programming interface operation for an application" (Tichenor, [0072] - Specialized components 430 can include augment creator 432, context tracker 434, context factor registrations 438, surface creator 440, and components and APIs which can be used for providing user interfaces, transferring data, and controlling the specialized components, such as interfaces 432. In some implementations, components 400 can be in a computing system that is distributed across multiple computing devices or can be an interface to a server-based application executing one or more of specialized components 430; [0122] - In example 1000, the artificial reality system identifies geometric surfaces by automatically locating flat surfaces of at least a certain size. In this manner, the artificial reality system has automatically identified surfaces 1102 and 1104. The artificial reality system also identified a surface on the floor but the user instructed (not shown) the system to divide that surface into surfaces 1110 and 1112. The user also caused (not shown) the artificial reality system to create surface 1108 by placing her hand on the surface. Existing real and virtual objects located on these surfaces are automatically added to them).
Claim 7 adds into claim 1 "wherein the at least one physical characteristic includes a depth, a distance, a direction, or a size of the object to support the command" (Tichenor, [0107] - Surfaces can automatically be assigned some properties such as size, orientation, shape, or meta-tags (based on object recognition), and users can define other properties which may be from a pre-defined set of property categories).
Claim 8 adds into claim 1 "identifying a gesture; wherein identifying the action is based on the application of the language model to the command, the image, and the gesture" (Tichenor, [0077] - Thus, surface creator 440 can define surfaces relative to the artificial reality system position, can identify surface geometries or object types specified for creating a surface, or can create surfaces relative to a user input (e.g., in response to a user performing an air tap, outlining a surface with a gesture, putting a hand, controller, or other peripheral device on the surface, defining a plane in the air with her hand to be used as a surface, attaching a surface to an object related to detected user interaction, etc.); [0125] - The user can further move the augment, e.g., by indicating a new surface with the users gaze, by performing a gesture (e.g., "dragging" the augment to a new location), with a voice command, etc.).
Claims 9-16, and 17-20 claim a computing apparatus and a computer-readable storage medium storing program instructions based on the method of claims 1-8; therefore, they are rejected under a similar rationale.
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHU K NGUYEN whose telephone number is (571)272-7645. The examiner can normally be reached M-F 8-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel F. Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHU K NGUYEN/ Primary Examiner, Art Unit 2616
Prosecution Insights