DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
This office action is in reply to Applicant’s Response dated 06/30/2026. Claims 1, 10 and 19 are amended. Claims 1-20 are pending in the application.
Response to Arguments
The Applicant argues (see page 8) that Powderly does not teach, at least, "the one or more criteria relating to at least one of a quality or a state of the at least one of the first data or the second data," as recited in claim 1…None of these constitute a quality or a state of the first data or the second data. A quality or state of data is an attribute of the data itself rather than in the external physical environment that the data describes. Powderly's contextual factors describe where the user is, what objects are nearby, and who the user is; they say nothing about the quality or state of the data streams that the system processes.
In response to the Applicant’s argument, the Examiner respectfully disagrees. The term “state” or “quality” recited in the above limitation is not defined in the limitation. Nowhere in the limitation requires or recites that the “state” or “quality” be an “attribute of the data itself rather than in the external physical environment that the data describes”.
Powderly teaches that the wearable system can filter or select the user interaction options or virtual objects to be presented on the user interface 1310 based on contextual information and the contextual information includes information associated with the user's environment, the user, objects in the user's environment, etc. Example contextual information includes the affordance of a physical object with which the options are associated, the user's environment (such as whether the environment is a home or an office environment), the user's characteristics, the user's current interactions with objects in the environment, the user's physiological state, the user's psychological state (Powderly, see figs. 8-10; see paragraph 0112). Under the broadest reasonable interpretation, contextual information including the affordance of a physical object with which the options are associated, the user's environment (such as whether the environment is a home or an office environment), the user's characteristics, the user's current interactions with objects in the environment, the user's physiological state and the user's psychological state are states or quality of the data in which the filtering or selection of the interaction options or virtual objects is based on. Thus, Powderly teaches "the one or more criteria relating to at least one of a quality or a state of the at least one of the first data or the second data," (Powderly, see figs. 8-10; see paragraph 0112).
The Applicant argues (see pages 9-10) that Krishnamurthy's filtering constraint tools do not constitute "modifying a number or order of a set of filters in an adaptive filter stack." Krishnamurthy does not disclose any system, module, or operation that dynamically modifies the number or order of its filters based on any criterion. There is no adaptive mechanism in Krishnamurthy that changes the number or order of those filter selections based on any data quality or state. Krishnamurthy's filtering criteria also have nothing to do with "the one or more criteria relating to at least one of a quality or a state of the at least one of the first data or the second data" as recited in the independent claims.
In response to the Applicant’s argument, the Examiner respectfully disagrees. The limitation that Krishnamurthy is relied upon to teach is “modifying a number or order of a set of filters in an adaptive filter stack based on an application of one or more criteria to at least one of the first data or the second data”. Krishnamurthy teaches that within the context of annotated dialogues being used as training examples, all instances of the find expression within the annotated dialogues of a set of training data are candidates for replacement by the replacement expression unless filtering constraint tools 516 are used to target a specific subset of the annotated dialogues or specific subregions thereof. One or more dialogue filters set via dialogue filters section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data to those annotated dialogues that pass the combination of dialogue filters and their associated criteria set via the dialogue filters section 524. The selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters (Krishnamurthy, see figs. 3 and 5; see paragraph 0071).
It is clear that Krishnamurthy teaches “modifying a number or order of a set of filters (selecting or setting dialogue filters/AND-ed dialogue filters) in an adaptive filter stack (dialogue filters (see fig. 5)) based on an application of one or more criteria (annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters) to at least one of the first data or the second data (annotated dialogues)” (Krishnamurthy, see figs. 3 and 5; see paragraph 0071).
The Applicant argues (see pages 8 and 10-11) that The Examiner additionally alleges, with respect to claims 10 and 19 as originally filed, that Powderly teaches "inferring an intent of a user based on an analysis of the at least one of the first data, the second data, and the third data through the modified adaptive filter stack…” and "Krishnamurthy equally fails to teach "inferring an intent of a user based on an analysis of the at least one of the first data, the second data, and the third data through the modified adaptive filter stack." The migration module has no function related to analyzing live data from physical input devices through a filter stack to infer a user's intent to perform an action in an XR environment.
In response to the Applicant’s arguments, the Applicant’s argument is regarding claims 10 and 19. The limitation "inferring an intent of a user based on an analysis of the at least one of the first data, the second data, and the third data through the modified adaptive filter stack" has been removed from claims 10 and 19. Therefore, the Applicant’s argument is moot.
The Applicant argues (see pages 11-12) that as amended, claims 1, 10, and 19 each further require that "the quality or state relates to a signal quality or signal fidelity of data received from the one or more input devices, the signal quality being determined by at least one of a signal strength, a signal to noise ratio, or a continuity of a received signal from the one or more input devices." Neither Powderly nor Krishnamurthy, alone or in combination, teaches or suggests this limitation.
In response to the Applicant’s argument, a new ground of rejection under 35 U.S.C. 103 is made in view of the amendments made to claims 1, 10 and 19.
The Applicant argues (see pages 12-13) that the paragraphs of Krishnamurthy cited in support of the Examiner's rationale do not establish any motivation to combine the references as proposed. Powderly and Krishnamurthy are directed to entirely different technical problems in entirely different technical fields. Incorporating Krishnamurthy's migration module into Powderly's AR system would not produce a system capable of "correctly recogniz[ing] inputs and improv[ing] intent recognition" because the migration module has no mechanism for doing either.
In response to the Applicant’s arguments, the Examiner respectfully disagrees. First, Powderly’s system does have a mechanism for recognizing inputs and intent recognition as will be discussed below. Second, for the sake of argument, even if Powderly’s system does not have the mechanism, incorporating Krishnamurthy's techniques in the system of Powderly would provide the mechanism (Krishnamurthy’s technique) to perform inputs and intent recognition and thereby, improving Powderly’s system.
Third, Powderly explicitly teaches that the user may say a word (e.g., saying “exit”) and/or actuate the user input device indicating the intent to close the virtual screen 1250. After receiving the indication, the ARD can stop projecting the screen 1250 on the table 1246 (Powderly, see paragraph 0107). Therefore, Powderly system does have a mechanism for recognizing inputs and intent recognition. Incorporating Krishnamurthy’s techniques in Powderly’s system would improve inputs and intent recognition.
Finally, in response to the Applicant’s argument that Powderly and Krishnamurthy are directed to entirely different technical problems in entirely different technical fields, it has been held that a prior art reference must either be in the field of the inventor’s endeavor or, if not, then be reasonably pertinent to the particular problem with which the inventor was concerned, in order to be relied upon as a basis for rejection of the claimed invention. See In re Oetiker, 977 F.2d 1443, 24 USPQ2d 1443 (Fed. Cir. 1992). In this case, as discussed above, Powderly explicitly teaches that the user may say a word (e.g., saying “exit”) and/or actuate the user input device indicating the intent to close the virtual screen 1250. Krishnamurthy teaches an example that seeks to illustrate the intricacies and complexities of capturing the intent of human speech within training data (Krishnamurthy, see paragraph 0015). The Applicant’s own invention requires “inferring an intent of a user based on an analysis of the at least one of the first data or the second data through the modified adaptive filter stack” (see claim 7 of the instant application). Clearly, all three, Powderly, Krishnamurthy and the Applicant’s invention address or concern capturing the intent of the user.
Additionally, Powderly is directed to techniques for automatically selecting or filtering available user interface interactions or virtual objects (Powderly, see abstract). Krishnamurthy is directed to process natural user interactions such as spoken or typed user utterances, without requiring the end user to use a particular syntax defined by a computer programming language; this allows end users to use natural language when addressing a computer (Krishnamurthy, see paragraph 0021 and fig. 5). Both Powderly and Krishnamurthy are directed to processing user interactions. Clearly, Powderly and Krishnamurthy are in the same field of endeavor. One of ordinary skill in the art would have been motivated to incorporate the techniques of Krishnamurthy in the system of Powderly in order to correctly recognize inputs and improve intent recognition (Krishnamurthy, see paragraphs 0015, 0109 and 0112)
The Applicant argues with respect to claims 3 and 12 (see page 14) that Nothing in paragraph 0035 -- or anywhere else in Krishnamurthy -- describes weighting or removing data based on input signal quality.
In response to the Applicant’s argument, the term “input signal quality” is not defined in the claims. Krishnamurthy teaches that a find expression 230 that is to be replaced within the set of previously-annotated training examples of initial training data 212, a replacement expression 232 that is to replace one or more instances of find expression 230 within the set of previously-identified training examples and Updated training data 216 that is output by migration module 140 includes a set of one or more updated annotated training examples, an example of which is depicted at 240. Migration module 140 may replace or augment some or all of input 222, of plan 224, and/or of action 226 based on migration data 230 to obtain and output updated training data 216 (Krishnamurthy, see paragraphs 0035-0036 and 0071). The previously-identified training examples or initial training data are input signals and are replaced based on their quality (previously-annotated or previously-identified or not being up-to-date) to produce updated training data. Thus, Krishnamurthy teaches weighting or removing data based on input signal quality (whether the inputs are previously-annotated or previously-identified or updated or not).
The Applicant argues (see pages 14-15) that claims 2, 9, 11, 17, and 20, each of which recites that the modifying of the number or order of the set of filters or the modifying of one or more properties of user interface elements is further based on an application of a machine-learned model. The claims recite a machine-learned model that modifies the number or order of filters in an adaptive filter stack based on criteria relating to the signal quality or fidelity of data received from physical input devices in a real-time XR environment, or that modifies properties of user interface elements based on inferred user intent.
In response to the Applicant’s argument, the Examiner respectfully disagrees. First, as the Applicant pointed out above, claims 2, 9, 11, 17, and 20 merely recite “wherein the modifying of the number or the order of the set of filters is further based on an application of a machine-learned model.” Claims 2, 9, 11, 17, and 20 do not recite or require a machine-learned model that modifies the number or order of filters in an adaptive filter stack based on criteria relating to the signal quality or fidelity of data received from physical input devices in a real-time XR environment, or that modifies properties of user interface elements based on inferred user intent.
As discussed above, it is clear that Krishnamurthy teaches “modifying a number or order of a set of filters (selecting or setting dialogue filters/AND-ed dialogue filters) in an adaptive filter stack (dialogue filters (see fig. 5)) based on an application of one or more criteria (annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters) to at least one of the first data or the second data (annotated dialogues)” (Krishnamurthy, see figs. 3 and 5; see paragraph 0071). Krishnamurthy further teaches that a machine learning model may be trained to recognize and/or recommend one or more of the find expression, replacement expression, and filtering constraint (e.g., based on training data including previous user inputs via the migration interface) and that updated subset of training examples is output by the computer program, which may be used for training a machine learning model (Krishnamurthy, see page 0042 and abstract). Krishnamurthy’s modifying is based on the training (application) of a machine learning model. Clearly, Krishnamurthy teaches “wherein the modifying of the number or the order of the set of filters is further based on an application of a machine-learned model” (Krishnamurthy, see page 0042 and abstract).
The Applicant argues (see pages 15-16) that Powderly's dwell-time mechanism says nothing about whether a user's gaze is angularly aligned with an element, and does not assess or use any measure of gaze divergence. The claimed gaze divergence filter functions as an input processing gate within the adaptive filter stack -- it allows or prevents input from being processed on a UI element with which a user is attempting to interact. Furthermore, claims 4 and 13 each depend from independent claims 1 and 10, respectively, which require the gaze divergence filter to be one of the filters within the claimed adaptive filter stack.
In response to the Applicant’s arguments, the Examiner respectfully disagrees. Claims 4 and 13 merely recite “wherein the set of filters includes a gaze divergence filter that takes into account an amount to which a user's gaze is focused upon at least one of the one or more user interface elements to allow or prevent input being processed on the at least one of the one or more user interface elements”. This limitation does not require or recite assessing or using any measure of gaze divergence or using the gaze divergence filter as an input processing gate. Furthermore, the limitation “to allow or prevent input being processed on the at least one of the one or more user interface elements” is merely an intended use or intended purpose.
Powderly teaches that eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input (Powderly, see paragraph 0096). Clearly, Powderly teaches “wherein the set of filters includes a gaze divergence filter that takes into account an amount to which a user's gaze is focused (gaze longer than a threshold time) upon at least one of the one or more user interface elements (particular real or virtual object) to allow or prevent input being processed on the at least one of the one or more user interface elements” (Powderly, see paragraph 0096).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Powderly et al. (U.S. PGPub 2017/0337742) in view of Krishnamurthy et al. (U.S. PGPub 2021/0224686) further in view of Menon (U.S. PGPub 2021/0409816).
Regarding claim 1, Powderly teaches A non-transitory computer-readable storage medium storing a set of instructions that, when executed by one or more computer processors, causes the one or more computer processors to perform operations, the operations comprising: accessing first data, the first data describing one or more user interface elements; (Powderly, see figs. 8-10; see paragraph 0004 an augmented reality display configured to present virtual content in a 3D view to a user; a pose sensor configured to acquire position or orientation data of a user and to analyze the position or orientation data to identify a pose of the user; and a hardware processor in communication with the pose sensor and the display....; see paragraph 0215 generating virtual content in a three-dimensional (3D) environment of a user...)
accessing second data, the second data being received from one or more input devices; and (Powderly, see figs. 8-10; see paragraphs 0004-0005 data acquired from a pose sensor to identify a pose of a user; identifying an interactable object in an 3D environment of the user based at least partly on the pose; receiving an indication to initiate an interaction with the interactable object;...; see paragraph 0215 analyzing data acquired from a pose sensor to identify a pose of a user; identifying an interactable object in an 3D environment of the user based at least partly on the pose...)
the one or more criteria relating to at least one of a quality or a state of the at least one of the first data or the second data, (Powderly, see figs. 8-10; see paragraph 0112 where filter or select the user interaction options or virtual objects to be presented on the user interface 1310 based on contextual information (criteria)... contextual information can include information associated with the user's environment, the user, objects in the user's environment, etc. Example contextual information can include the affordance of a physical object with which the options are associated, the user's environment (such as whether the environment is a home or an office environment), the user's characteristics, the user's current interactions with objects in the environment, the user's physiological state, the user's psychological state...)
However, Powderly does not explicitly teach modifying a number or order of a set of filters in an adaptive filter stack based on an application of one or more criteria to at least one of the first data or the second data,
Krishnamurthy teaches modifying a number or order of a set of filters in an adaptive filter stack based on an application of one or more criteria to at least one of the first data or the second data, (Krishnamurthy, see figs. 3 and 5; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters (filter stack) section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters (filter stack) are applied includes a preamble of the annotated dialogue)
It would have been obvious to one of ordinary skill in the art, at the time the invention was filed, to combine Powderly and Krishnamurthy to provide the technique of modifying a number or order of a set of filters in an adaptive filter stack based on an application of one or more criteria to at least one of the first data or the second data of Krishnamurthy in the system of Powderly in order to correctly recognize inputs and improve intent recognition (Krishnamurthy, see paragraphs 0015, 0109 and 0112).
However, Powderly-Krishnamurthy does not explicitly teach wherein the quality or state relates to a signal quality or signal fidelity of data received from the one or more input devices, the signal quality being determined by at least one of a signal strength, a signal to noise ratio, or a continuity of a received signal from the one or more input devices.
Menon teaches wherein the quality or state relates to a signal quality or signal fidelity of data received from the one or more input devices, the signal quality being determined by at least one of a signal strength, a signal to noise ratio, or a continuity of a received signal from the one or more input devices. (Menon, see paragraph 0018 converter device 205 selects OTT 235 before selecting other input sources...the converter device 205 selects the input source with the highest SNR of a list of preferred input sources. For example, when a user selects to watch a news broadcast, the converter device 205 selects input source wireless 245 (e.g., 5G) to provide the content of the news broadcast based on the input source wireless 245 having the highest SNR of the list of preferred input sources...)
It would have been obvious to one of ordinary skill in the art, at the time the invention was filed, to combine Powderly-Krishnamurthy and Menon to provide the technique of the quality or state relates to a signal quality or signal fidelity of data received from the one or more input devices, the signal quality being determined by at least one of a signal strength, a signal to noise ratio, or a continuity of a received signal from the one or more input devices of Menon in the system of Powderly-Krishnamurthy in order to improve viewing experience and mitigate service interruption (Menon, see paragraph 0019).
Regarding claim 2, Powderly-Krishnamurthy-Menon teaches wherein the modifying of the number or the order of the set of filters is further based on an application of a machine-learned model. (Krishnamurthy, see figs. 3 and 5; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters (filter stack) section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters (filter stack) are applied includes a preamble of the annotated dialogue; see paragraph 0042 filtering constraint includes receiving user input via a migration interface...a machine learning model may be trained...) The motivation regarding to the obviousness to claim 1 is also applied to claim 2.
Regarding claim 3, Powderly-Krishnamurthy-Menon teaches wherein the accessing of the first data or the second data comprises filtering or modifying the first data or the second data via a dynamic continuous input system before providing the first data or the second data to the adaptive filter stack, (Krishnamurthy, see figs. 3; see paragraph 0035 a find expression 230 that is to be replaced within the set of previously-annotated training examples of initial training data 212, a replacement expression 232 that is to replace one or more instances of find expression 230 within the set of previously-identified training examples...; see also paragraph 0071)
wherein the filtering or modifying includes weighting or removing at least some of the first data or the second data based on input signal quality. (Krishnamurthy, see figs. 3; see paragraph 0035 a find expression 230 that is to be replaced within the set of previously-annotated training examples of initial training data 212, a replacement expression 232 that is to replace one or more instances of find expression 230 within the set of previously-identified training examples...; see also paragraph 0071) The motivation regarding to the obviousness to claim 1 is also applied to claim 3.
Regarding claim 4, Powderly-Krishnamurthy-Menon teaches wherein the set of filters includes a gaze divergence filter that takes into account an amount to which a user's gaze is focused upon at least one of the one or more user interface elements to allow or prevent input being processed on the at least one of the one or more user interface elements. (Powderly, see paragraph 0096 eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes can be tracked and an accommodation/vergence model can be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing...)
Regarding claim 5, Powderly-Krishnamurthy-Menon teaches the operations further comprising accessing third data, the third data received from one or more environmental sensors, the third data characterizing the second data, and (Powderly, see figs. 8-10; see paragraph 0004 an augmented reality display configured to present virtual content in a 3D view to a user; a pose sensor configured to acquire position or orientation data of a user and to analyze the position or orientation data to identify a pose of the user; and a hardware processor in communication with the pose sensor and the display....; see paragraph 0215 generating virtual content in a three-dimensional (3D) environment of a user...)
wherein the modifying of the number or order of the set of filters in the adaptive filter stack is further based on the application of the one or more criteria to the third data. (Krishnamurthy, see figs. 3 and 5; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters (filter stack) section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters (filter stack) are applied includes a preamble of the annotated dialogue) The motivation regarding to the obviousness to claim 1 is also applied to claim 5.
Regarding claim 6, Powderly-Krishnamurthy-Menon teaches wherein the third data comprises at least one of a position of at least one of the one or more input devices, a gaze angle, a gaze direction, a velocity, a continuous input duration, a trajectory, or a field of view. (Powderly, see paragraph 0096 eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes can be tracked and an accommodation/vergence model can be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing...)
Regarding claim 7, Powderly-Krishnamurthy-Menon teaches the operations further comprising: inferring an intent of a user based on an analysis of the at least one of the first data or the second data through the modified adaptive filter stack; and (Krishnamurthy, see figs. 3 and 5; see paragraph 0022 user may interact with a computer using natural language by asking the computer to answer a question (intent of a user) or by providing a command to the computer. In response, the conversational computing system is trained to automatically perform one or more actions, such as answering questions or otherwise assisting a user (e.g., reporting today's weather forecast in response to the human saying or typing “what is the weather like?”).; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters are applied includes a preamble of the annotated dialogue)
based on the inferring of the intent, modifying one or more properties of the one or more user interface elements based on one or more preconfigured rules or an application of a machine-learned model. (Krishnamurthy, see figs. 3 and 5; see paragraph 0022 user may interact with a computer using natural language by asking the computer to answer a question (intent of a user) or by providing a command to the computer. In response, the conversational computing system is trained to automatically perform one or more actions, such as answering questions or otherwise assisting a user (e.g., reporting today's weather forecast in response to the human saying or typing “what is the weather like?”).; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters are applied includes a preamble of the annotated dialogue) The motivation regarding to the obviousness to claim 1 is also applied to claim 7.
Regarding claim 8, Powderly-Krishnamurthy-Menon teaches wherein the modifying of the one or more properties of the one or more user interface elements includes at least one of enabling, disabling, or reshaping the one or more user interface elements. (Powderly, see fig. 8; see paragraph 0082 rendering virtual content in relation to recognized objects. The method 800 describes how a virtual scene may be represented to a user of the wearable system. The user may be geographically remote from the scene. For example, the user may be New York, but may want to view a scene that is presently going on in California, or may want to go on a walk with a friend who resides in California.; see paragraph 0083 the desired virtual scene may be accordingly displayed to the user at block 850. For example, the desired virtual scene (e.g., user in CA) may be displayed at the appropriate orientation, position, etc., in relation to the various objects and other surroundings of the user in New York...)
Regarding claim 9, Powderly-Krishnamurthy-Menon teaches wherein the modifying of the one or more properties is further based on an application of a machine-learned model. (Krishnamurthy, see figs. 3 and 5; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters (filter stack) section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters (filter stack) are applied includes a preamble of the annotated dialogue; see paragraph 0042 filtering constraint includes receiving user input via a migration interface...a machine learning model may be trained...) The motivation regarding to the obviousness to claim 1 is also applied to claim 9.
Regarding claims 10 and 19, Powderly teaches A system comprising: one or more computer processors; one or more computer memories; and a set of instructions stored in the one or more computer memories, the set of instructions configuring the one or more computer processors to perform operations, the operations comprising: accessing first data, the first data describing one or more user interface elements; (Powderly, see figs. 8-10; see paragraph 0004 an augmented reality display configured to present virtual content in a 3D view to a user; a pose sensor configured to acquire position or orientation data of a user and to analyze the position or orientation data to identify a pose of the user; and a hardware processor in communication with the pose sensor and the display....; see paragraph 0215 generating virtual content in a three-dimensional (3D) environment of a user...)
accessing second data, the second data received from one or more environmental sensors; (Powderly, see figs. 8-10; see paragraphs 0004-0005 data acquired from a pose sensor to identify a pose of a user; identifying an interactable object in an 3D environment of the user based at least partly on the pose; receiving an indication to initiate an interaction with the interactable object;...; see paragraph 0215 analyzing data acquired from a pose sensor to identify a pose of a user; identifying an interactable object in an 3D environment of the user based at least partly on the pose...)
accessing third data, the third data being received from one or more input devices; (Powderly, see figs. 8-10; see paragraph 0087 various inputs may be utilized... inputs to determine that the user of the system is playing a monster battling game with one or more monsters at various locations, monsters dying or running away under various conditions (such as if the user shoots the monster), walls or other objects at various locations...; see paragraph 0088 Controls or inputs from the user are another input to the wearable system 900. As described herein, user inputs can include visual input, gestures, totems, audio input, sensory input (such as e.g., physiological data acquired by sensors 232 in FIG. 2), etc.)
the one or more criteria relating to at least one of a quality or a state of the at least one of the first data, the second data, or the third data; (Powderly, see figs. 8-10; see paragraph 0112 where filter or select the user interaction options or virtual objects to be presented on the user interface 1310 based on contextual information (criteria)... contextual information can include information associated with the user's environment, the user, objects in the user's environment, etc. Example contextual information can include the affordance of a physical object with which the options are associated, the user's environment (such as whether the environment is a home or an office environment), the user's characteristics, the user's current interactions with objects in the environment, the user's physiological state, the user's psychological state...)
However, Powderly does not explicitly teach modifying a number or order of a set of filters in an adaptive filter stack based on an application of one or more criteria to at least one of the first data, the second data, or the third data,
Krishnamurthy teaches modifying a number or order of a set of filters in an adaptive filter stack based on an application of one or more criteria to at least one of the first data, the second data, or the third data, (Krishnamurthy, see figs. 3 and 5; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters (filter stack) section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters (filter stack) are applied includes a preamble of the annotated dialogue)
It would have been obvious to one of ordinary skill in the art, at the time the invention was filed, to combine Powderly and Krishnamurthy to provide the technique of modifying a number or order of a set of filters in an adaptive filter stack based on an application of one or more criteria to at least one of the first data, the second data, or the third data of Krishnamurthy in the system of Powderly in order to correctly recognize inputs and improve intent recognition (Krishnamurthy, see paragraphs 0015, 0109 and 0112).
However, Powderly-Krishnamurthy does not explicitly teach wherein the quality or state relates to a signal quality or signal fidelity of data received from the one or more input devices, the signal quality being determined by at least one of a signal strength, a signal to noise ratio, or a continuity of a received signal from the one or more input devices.
Menon teaches wherein the quality or state relates to a signal quality or signal fidelity of data received from the one or more input devices, the signal quality being determined by at least one of a signal strength, a signal to noise ratio, or a continuity of a received signal from the one or more input devices. (Menon, see paragraph 0018 converter device 205 selects OTT 235 before selecting other input sources...the converter device 205 selects the input source with the highest SNR of a list of preferred input sources. For example, when a user selects to watch a news broadcast, the converter device 205 selects input source wireless 245 (e.g., 5G) to provide the content of the news broadcast based on the input source wireless 245 having the highest SNR of the list of preferred input sources...)
It would have been obvious to one of ordinary skill in the art, at the time the invention was filed, to combine Powderly-Krishnamurthy and Menon to provide the technique of the quality or state relates to a signal quality or signal fidelity of data received from the one or more input devices, the signal quality being determined by at least one of a signal strength, a signal to noise ratio, or a continuity of a received signal from the one or more input devices of Menon in the system of Powderly-Krishnamurthy in order to improve viewing experience and mitigate service interruption (Menon, see paragraph 0019).
Regarding claims 11 and 20, Powderly-Krishnamurthy-Menon teaches wherein the modifying of the number or the order of the set of filters is further based on an application of a machine-learned model. (Krishnamurthy, see figs. 3 and 5; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters (filter stack) section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters (filter stack) are applied includes a preamble of the annotated dialogue; see paragraph 0042 filtering constraint includes receiving user input via a migration interface...a machine learning model may be trained...) The motivation regarding to the obviousness to claims 10 and 19 is also applied to claims 11 and 20.
Regarding claim 12, Powderly-Krishnamurthy-Menon teaches wherein the accessing of the first data or the second data comprises filtering or modifying the first data or the second data via a dynamic continuous input system before providing the first data or the second data to the adaptive filter stack, (Krishnamurthy, see figs. 3; see paragraph 0035 a find expression 230 that is to be replaced within the set of previously-annotated training examples of initial training data 212, a replacement expression 232 that is to replace one or more instances of find expression 230 within the set of previously-identified training examples...; see also paragraph 0071)
wherein the filtering or modifying includes weighting or removing at least some of the first data or the second data based on input signal quality. (Krishnamurthy, see figs. 3; see paragraph 0035 a find expression 230 that is to be replaced within the set of previously-annotated training examples of initial training data 212, a replacement expression 232 that is to replace one or more instances of find expression 230 within the set of previously-identified training examples...; see also paragraph 0071) The motivation regarding to the obviousness to claims 10 and 19 is also applied to claim 12.
Regarding claim 13, Powderly-Krishnamurthy-Menon teaches wherein the set of filters includes a gaze divergence filter that takes into account an amount to which a user's gaze is focused upon at least one of the one or more user interface elements to allow or prevent input being processed on the at least one of the one or more user interface elements. (Powderly, see paragraph 0096 eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes can be tracked and an accommodation/vergence model can be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing...)
Regarding claim 14, Powderly-Krishnamurthy-Menon teaches wherein the third data comprises at least one of a position of at least one of the one or more input devices, a gaze angle, a gaze direction, a velocity, a continuous input duration, a trajectory, or a field of view. (Powderly, see paragraph 0096 eyes to look at different real or virtual objects in the user's FOR. If the user's gaze at a particular real or virtual object is longer than a threshold time, the real or virtual object may be selected as the user input. In some implementations, the vergence of the user's eyes can be tracked and an accommodation/vergence model can be used to determine the accommodation state of the user's eyes, which provides information on a depth plane on which the user is focusing...)
Regarding claim 15, Powderly-Krishnamurthy-Menon teaches the operations further comprising accessing third data, the third data received from one or more environmental sensors, the third data characterizing the second data, and (Powderly, see figs. 8-10; see paragraph 0004 an augmented reality display configured to present virtual content in a 3D view to a user; a pose sensor configured to acquire position or orientation data of a user and to analyze the position or orientation data to identify a pose of the user; and a hardware processor in communication with the pose sensor and the display....; see paragraph 0215 generating virtual content in a three-dimensional (3D) environment of a user...)
wherein the modifying of the number or order of the set of filters in the adaptive filter stack is further based on the application of the one or more criteria to the third data. (Krishnamurthy, see figs. 3 and 5; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters (filter stack) section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters (filter stack) are applied includes a preamble of the annotated dialogue) The motivation regarding to the obviousness to claims 10 and 19 is also applied to claim 15.
Regarding claim 16, Powderly-Krishnamurthy-Menon teaches wherein the modifying of the one or more properties of the one or more user interface elements includes at least one of enabling, disabling, or reshaping the one or more user interface elements. (Powderly, see fig. 8; see paragraph 0082 rendering virtual content in relation to recognized objects. The method 800 describes how a virtual scene may be represented to a user of the wearable system. The user may be geographically remote from the scene. For example, the user may be New York, but may want to view a scene that is presently going on in California, or may want to go on a walk with a friend who resides in California.; see paragraph 0083 the desired virtual scene may be accordingly displayed to the user at block 850. For example, the desired virtual scene (e.g., user in CA) may be displayed at the appropriate orientation, position, etc., in relation to the various objects and other surroundings of the user in New York...)
Regarding claim 17, Powderly-Krishnamurthy-Menon teaches wherein the modifying of the one or more properties is further based on an application of a machine-learned model. (Krishnamurthy, see figs. 3 and 5; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters (filter stack) section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters (filter stack) are applied includes a preamble of the annotated dialogue; see paragraph 0042 filtering constraint includes receiving user input via a migration interface...a machine learning model may be trained...) The motivation regarding to the obviousness to claims 10 and 19 is also applied to claim 15.
Regarding claim 18, Powderly-Krishnamurthy-Menon teaches the operations further comprising: inferring an intent of a user based on an analysis of the at least one of the first data or the second data through the modified adaptive filter stack; and (Krishnamurthy, see figs. 3 and 5; see paragraph 0022 user may interact with a computer using natural language by asking the computer to answer a question (intent of a user) or by providing a command to the computer. In response, the conversational computing system is trained to automatically perform one or more actions, such as answering questions or otherwise assisting a user (e.g., reporting today's weather forecast in response to the human saying or typing “what is the weather like?”).; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters are applied includes a preamble of the annotated dialogue)
based on the inferring of the intent, modifying one or more properties of the one or more user interface elements based on one or more preconfigured rules or an application of a machine-learned model. (Krishnamurthy, see figs. 3 and 5; see paragraph 0022 user may interact with a computer using natural language by asking the computer to answer a question (intent of a user) or by providing a command to the computer. In response, the conversational computing system is trained to automatically perform one or more actions, such as answering questions or otherwise assisting a user (e.g., reporting today's weather forecast in response to the human saying or typing “what is the weather like?”).; see paragraph 0071 implementation of filtering constraint tools 516... one or more dialogue filters set via dialogue filters section 524 are applied first by the migration module to filter all annotated dialogues of the set of training data...the selected dialogue filters may be AND-ed together by the migration module to require that the annotated dialogues targeted for migration pass the criteria associated with all of the selected dialogue filters...which a second subset of the filters are applied includes a preamble of the annotated dialogue)
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MENG VANG whose telephone number is (571)270-7023. The examiner can normally be reached M-F 8AM-2PM, 3PM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, NICHOLAS TAYLOR can be reached at (571) 272-3889. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MENG VANG/Primary Examiner, Art Unit 2443
