DETAILED ACTION
This FINAL action is in response to Application No. 18/669,877 filed 5/21/2024. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . The amendment presented on 6/3/2026 which provides amendments to claims 1, 9-16, and 20, and new claim 21, is hereby acknowledged. Claims 1-21 are currently pending.
Claim Objections - Withdrawn
The previous objections to claims 10-15 are withdrawn as necessitated by amendment.
Claim Rejections – Withdrawn
The previous rejections of claims 10 and 15 as under 35 U.S.C. 112 are withdrawn as necessitated by amendment.
Claim Interpretation – Withdrawn
The previous interpretation of claims 10 and 15 as invoking 35 U.S.C. 112(f) is withdrawn as necessitated by amendment.
Response to Arguments
Applicant's arguments with respect to the previous prior art rejections of claims 1-7 and 9-20 have been considered, however, the amendment(s) to the claims necessitated a new consideration and search resulting in new prior art cited below.
Claim Rejections - 35 USC § 112
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim 21 is rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claim 21, “automatically adjusting the location of the selected region is triggered in response to the velocity of drift exceeding a threshold” does not appear disclosed within the specification. Specifically, the specification at [0037] describes:
“In one embodiment, the velocity of the drift may be also factored in to assess whether the user’s finger is moving away from the target area quickly or slowly over time, where the finger straying swiftly may indicate a lapse in concentration, and a gradual shift may suggest muscle fatigue.”
This does not disclose using a threshold, rather, this discloses reasons for swift or gradual drift and appears to imply that any of the reasons could be used for adjusting the regions. The specification appears to perform the adjusting no matter what the velocity of drift is, e.g., the specification does not explicitly or implicitly indicate that adjusting the regions is only for one of swift drift or gradual drift (see also specification at [0031] and [0036] as describing adjustment based on “gradual” shift, but also adjusting for “critical moments where every second counts”).
Therefore, claim 21 appears to fail the written description requirement of 35 U.S.C. §112(a).
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1-7 and 9-20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu et a. (US 2013/0222247 A1, hereinafter “Liu”), and further in view of Peng (US 2024/0176483 A1).
Regarding claim 1, Liu teaches a computer-implemented method, comprising:
monitoring a plurality of positions at which a user provides input to perform a specific action over time, the specific action associated with a selected region of a user interface. More specifically, steps 508 to 510 pertain to the continuous monitoring of input to character input keys (specific actions) of a portion (selected region) of a virtual keyboard until a minimum number of key selections have been received since the last adjustment of the keyboard (over time) (Liu, abstract, Figure 5A, [0054]-[0055]). Offset vectors (plurality of positions) that define the distance and direction the input (user provided input) is from center of a particular key are determined in step 512 (Liu, Figure 5A, [0056]).
determining a drift pattern…with respect to the plurality of positions. More specifically, the offset vectors (plurality of positions) for each key are then used to determine an average offset (drift pattern) (Liu, Figure 5A, step 512, [0056]).
based at least on the drift pattern, automatically adjusting a location of the selected region of the user interface, associated with the specific action, wherein further input provided by the user at subsequent positions corresponding to the drift pattern is able to be registered as input to perform the specific action. More specifically, at least position, size, spacing, and rotation of the particular key (specific action) of the virtual keyboard is adjusted based on the average offset (drift pattern) (Liu, steps 514-522, Figure 5A, [0057]-[0060]).
However, Liu may not explicitly teach every aspect of
[the drift pattern] includes temporal shifts.
Peng discloses techniques for implementing a virtualized physical controller. The techniques may include receiving user input(s) on a touchscreen display of a user device within input regions. The positions and shapes of the input regions may be iteratively redefined as touch input is monitored on the touchscreen display (Peng abstract). The input regions are automatically adjusted at various points in time by aggregating usage data (places on a touchscreen display that a user has touched when interacting with a particular input region) over a period of time. “As a user grows, the placement of that user's hands on a user device may change and the locations of various regions may also need to be updated” (Peng, Figure 6, [0051], [0063]-[0068], construed as determining a drift pattern including temporal shifts and adjusting the input regions accordingly).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention given the teachings of Liu and Peng that a method for automatically adjusting a user interface for a touch screen when input drift is detected would include [the drift pattern] includes temporal shifts. With both Liu and Peng disclosing determining that touch user interface controls should be adjusted when touch input is not centered on the touch user interface controls, and with Peng additionally disclosing the monitoring of the touch input is over a period of time and determining to adjust the touch user interface controls is also over time (e.g., as a user grows), one of ordinary skill in the art of implementing a method for automatically adjusting a user interface for a touch screen when input drift is detected would include [the drift pattern] includes temporal shifts in order to continuously prevent erroneous input by ensuring touch user interface controls reside in the most appropriate location as a user grows. One would therefore be motivated to combine these teachings as in doing so would create this method for automatically adjusting a user interface for a touch screen when input drift is detected.
Regarding claim 2, Liu and Peng teach the computer-implemented method of claim 1, wherein the user input is provided using a touch screen, and wherein the touch screen does not provide tactile feedback associated with the selected region. More specifically, while a physical keyboard provides tactile feedback in the form of keyboard curvature, discrete key caps, and key actuation clicks, touch displays generally lack these physical cues. As a result, the user's hands and fingers may drift while typing on a virtual keyboard (Liu, [0016], [0021]). (See also Peng, [0002], [0004]).
Regarding claim 3, Liu and Peng teach the computer-implemented method of claim 1, wherein a magnitude of the adjusting is based in part on a determined size of the touch screen or space for the user interface. More specifically, to ensure that the boundary of a particular key does not overlap the boundary of an adjacent key (space for the user interface), the adjustment of each key may be limited to a predetermined distance from the home position of the key or limited based on the position of the adjacent keys (Liu, [0032]).
Regarding claim 4, Liu and Peng teach the computer-implemented method of claim 1, further comprising: receiving control data for controlling a magnitude to which the location is adjusted. More specifically, adjusting instructions may calculate the average of 10 vectors and shift the position of a key in the direction of the average offset vector by the magnitude of the average vector up to a given maximum shift (Liu, [0033]).
Regarding claim 5, Liu and Peng teach the computer-implemented method of claim 1, wherein the selected region is associated with other input regions corresponding to related actions, and wherein adjusting the location of the selected region further comprises adjusting locations of at least a subset of the other input regions. More specifically, Figure 5B defines a process similar to the process of Figure 5A, however, a grouping of keys of the virtual keyboard (selected region is associated with other input regions corresponding to related actions) are adjusted together by at least position, size, spacing, and rotation (Liu, Figure 5B, [0061]-[0069]).
Regarding claim 6, Liu and Peng teach the computer-implemented method of claim 1, further comprising: detecting the input is provided using both a left hand and right hand; detecting different drift patterns for the left hand and the right hand, wherein the drift patterns are used in adjusting the location of at least the selected region. More specifically, Figure 1 depicts how portions of a virtual keyboard are separately adjusted for each of the left and right hands’ drifted positions (Liu, Figure 1, [0019]-[0022]).
Regarding claim 7, Liu and Peng teach the computer-implemented method of claim 1, wherein the input corresponds to touch, gesture, or motion input. More specifically, the input is touch on a touchscreen (Liu, abstract, [0001]) (See also Peng, abstract).
Regarding claim 9, Liu teaches at least one processor comprising: one or more processing units to:
monitor at least one position at which a user provides input to perform a specific action, the specific action associated with a selected region of a user interface. More specifically, steps 508 to 510 pertain to the continuous monitoring of input to character input keys (specific actions) of a portion (selected region) of a virtual keyboard until a minimum number of key selections have been received since the last adjustment of the keyboard (over time) (Liu, abstract, Figure 5A, [0054]-[0055]). Offset vectors (at least one position) that define the distance and direction the input (user provided input) is from center of a particular key are determined in step 512 (Liu, Figure 5A, [0056]).
determine a drift pattern… with respect to the at least one position. More specifically, the offset vectors (at least one position) for each key are then used to determine an average offset (drift direction) (Liu, Figure 5A, step 512, [0056]).
provide an adjustment value to be applied to a location of the selected region of the user interface, associated with the specific action, according to the drift pattern, wherein further input provided by the user at at least one subsequent position corresponding to the drift pattern is able to be registered as input to perform the specific action. More specifically, at least position, size, spacing, and rotation of the particular key (specific action) or portion of keys of the virtual keyboard is adjusted based on the average offset (drift pattern) (Liu, steps 514-522, Figure 5A, [0057]-[0060; Figure 5B, [0061]-[0069]).
However, Liu may not explicitly teach every aspect of
[the drift pattern] includes temporal shifts.
Peng discloses techniques for implementing a virtualized physical controller. The techniques may include receiving user input(s) on a touchscreen display of a user device within input regions. The positions and shapes of the input regions may be iteratively redefined as touch input is monitored on the touchscreen display (Peng abstract). The input regions are automatically adjusted at various points in time by aggregating usage data (places on a touchscreen display that a user has touched when interacting with a particular input region) over a period of time. As a user grows, the placement of that user's hands on a user device may change and the locations of various regions may also need to be updated (Peng, Figure 6, [0051], [0063]-[0068], construed as determining a drift pattern including temporal shifts and adjusting the input regions accordingly).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention given the teachings of Liu and Peng that a processor for adjusting a user interface for a touch screen when input drift is detected would include [the drift pattern] includes temporal shifts. With both Liu and Peng disclosing determining that touch user interface controls should be adjusted when touch input is not centered on the touch user interface controls, and with Peng additionally disclosing the monitoring of the touch input is over a period of time and determining to adjust the touch user interface controls is also over time (e.g., as a user grows), one of ordinary skill in the art of implementing a processor for adjusting a user interface for a touch screen when input drift is detected would include [the drift pattern] includes temporal shifts in order to continuously prevent erroneous input by ensuring touch user interface controls reside in the most appropriate location as a user grows. One would therefore be motivated to combine these teachings as in doing so would create this processor for adjusting a user interface for a touch screen when input drift is detected.
Regarding claim 10, Liu and Peng teach the at least one processor of claim 9, wherein a user interface is implemented using a touch screen, and wherein the touch screen does not provide tactile feedback associated with the selected region. More specifically, while a physical keyboard provides tactile feedback in the form of keyboard curvature, discrete key caps, and key actuation clicks, touch displays generally lack these physical cues. As a result, the user's hands and fingers may drift while typing on a virtual keyboard (Liu, [0016], [0021]). (See also Peng, [0002], [0004]).
Regarding claim 11, Liu and Peng teach the at least one processor of claim 10, wherein a magnitude of the adjusting is based in part on a determined size of the touch screen or space for the user interface. More specifically, to ensure that the boundary of a particular key does not overlap the boundary of an adjacent key (space for the user interface), the adjustment of each key may be limited to a predetermined distance from the home position of the key or limited based on the position of the adjacent keys (Liu, [0032]).
Regarding claim 12 Liu and Peng teach the at least one processor of claim 9, wherein the selected region is associated with other input regions corresponding to related actions, and wherein adjusting the location of the selected region further comprises adjusting locations of at least a subset of the other input regions. More specifically, Figure 5B defines a process similar to the process of Figure 5A, however, a grouping of keys of the virtual keyboard (selected region is associated with other input regions corresponding to related actions) are adjusted together by at least position, size, spacing, and rotation (Liu, Figure 5B, [0061]-[0069]).
Regarding claim 13, Liu and Peng teach the at least one processor of claim 9, wherein input is provided using both a left hand and a right hand, and wherein the one or more processing units are further to detect different drift patterns for the left hand and the right hand, wherein the different drift patterns are used in adjusting the location of at least the selected region. More specifically, Figure 1 depicts how portions of a virtual keyboard are separately adjusted for each of the left and right hands’ drifted positions (Liu, Figure 1, [0019]-[0022]).
Regarding claim 14, Liu and Peng teach the at least one processor of claim 9, wherein the input corresponds to touch, gesture, or motion input. More specifically, the input is touch on a touchscreen (Liu, abstract, [0001]). (See also Peng, abstract).
Regarding claim 15, Liu and Peng teach the at least one processor of claim 9, wherein the at least one processor is included in a system configured for one or more of: general simulation operations; specific simulation operations to test or validate autonomous machine applications; digital twin operations; light transport simulation; graphical output; deep learning operations; edge device operations; virtual reality (VR) content generation or presentation; augmented reality (AR) content generation or presentation; mixed reality (MR) content generation or presentation; one or more Virtual Machines (VMs) operation; a data center operations; hardware testing by simulation; synthetic data generation; generative Artificial Intelligence (AI) operations; one or more large language model (LLMs); one or more vision language model (VLMs); a collaborative content creation platform for 3D assets; or cloud computing resources. More specifically, Liu describes an input mechanism of a virtual keyboard, which is construable to be a simulation of input with a physical keyboard (system for performing simulation operations). Displaying the input from typing with the virtual keyboard is construable to be a system for rendering graphical output (Liu, [0016]-[0017]).
Regarding claims 16-20, these claims recite the system that substantially performs the steps of performed by the at least one processor of claims 9-12 and 15, therefore, the same rationale of rejection is applicable.
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu and Peng, and further in view of Yoshizawa et al. (US 2021/0011605 A1, hereinafter “Yoshizawa”).
Regarding claim 8, Liu and Peng teach the computer-implemented method of claim 1, however, may not explicitly teach every aspect of wherein the drift pattern is monitored by an external motion capturing device.
Yoshizawa discloses a personal digital assistant includes a display control unit that causes an object as a target of the touch operation to be displayed in the image display unit. The display control unit causes the object to be displayed based on a relative shaking amount of the operation instruction object (finger) such that the object is shifted in accordance with the relative shaking amount (Yoshizawa, abstract). Figures 9A and 9B depict a head mounted external camera that can be used in accordance with the process of Figure 5A where a camera is used to monitor a operation instruction object (finger) to calculate a relative shaking amount due to the user riding in a rocking vehicle (construable as a drift pattern) and shift the user interface accordingly (Yoshizawa, Figure 5A, 9A, and 9B, at least steps 104-108, [0071]-[0073], [0112]-[0115]).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention given the teachings of Liu and Peng with Yoshizawa that a method for automatically adjusting a user interface for a touch screen when input drift is detected would include wherein the drift is monitored by an external motion capturing device. With Liu, Peng, and Yoshizawa disclosing determining that touch input is not centered on touch user interface controls and adjusting the touch user interface controls accordingly, and with Yoshizawa additionally disclosing monitoring the input drift from an external camera, one of ordinary skill in the art of implementing a method of method for automatically adjusting a user interface for a touch screen when input drift is detected would include wherein the drift is monitored by an external motion capturing device in order to supplement touch detected drift with information from the point of view of the user’s eyes for even better drift adjustment and also to have the option of shifting the touch user interface controls to the proper location before the touch input is received, ultimately limiting erroneous input. One would therefore be motivated to combine these teachings as in doing so would create this method for automatically adjusting a user interface for a touch screen when input drift is detected.
Claim(s) 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Liu and Peng, and further in view of Carney et al. (US 9,092,407 B2, hereinafter “Carney”).
Regarding claim 21, Liu and Peng teach the computer-implemented method of claim 1, however, may not explicitly teach every aspect of wherein determining the drift pattern comprises determining a velocity of drift based on changes in the plurality of positions over time, and wherein automatically adjusting the location of the selected region is triggered in response to the velocity of drift exceeding a threshold.
Carney discloses a virtual interface adjustment system that detects, based on user input data associated with user input received by way of a virtual interface of a user device, an erroneous input pattern that includes at least one unintentional input, determining, based on the erroneous input pattern, an adjustment procedure configured to adjust the virtual interface to avert future unintentional inputs (Carney, abstract). Input pattern facility may detect an erroneous input pattern of touch locations that consist unintentional inputs occurring at or above a predefined frequency threshold (Carney, col 4, line 46 – col 6, line 20, the unintentional inputs are inputs that drifted away from intended input regions and adjusting the virtual interface when they occur at or above a predefined frequency (rate/velocity over time) threshold is construed as adjusting the location of the selected region is triggered in response to the velocity of drift exceeding a threshold).
It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention given the teachings of Liu and Peng with Carney that a method for automatically adjusting a user interface for a touch screen when an input drift pattern is detected would include wherein determining the drift pattern comprises determining a velocity of drift based on changes in the plurality of positions over time, and wherein automatically adjusting the location of the selected region is triggered in response to the velocity of drift exceeding a threshold. With Liu, Peng, and Carney disclosing determining that touch input is not centered on touch user interface controls and adjusting the touch user interface controls accordingly, and with Carney additionally disclosing determining the frequency of drift occurrences exceeds a threshold before adjusting the user interface, one of ordinary skill in the art of implementing a method for automatically adjusting a user interface for a touch screen when an input drift pattern is detected would include wherein determining the drift pattern comprises determining a velocity of drift based on changes in the plurality of positions over time, and wherein automatically adjusting the location of the selected region is triggered in response to the velocity of drift exceeding a threshold in order to ensure the user interface adjustment is actually needed, avoiding reacting to one-off anomalous input misses. One would therefore be motivated to combine these teachings as in doing so would create this method for automatically adjusting a user interface for a touch screen when input drift is detected.
Pertinent Prior Art
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider these references fully when responding to this action.
Winer (US 2015/0346905 A1) – adjusting user interface regions based on a erroneous touch inputs.
Arnold (US 2013/0019191 A1) – adjusting user interface regions based on touch inputs.
McAleer (US 2012/0166995 A1) – adjusting user interface regions based on touch inputs.
Kairls (US 2004/0178994 A1) – adjusting user interface regions based on touch inputs.
Cantrell (US 2014/0195923 A1) – adjusting user interface regions based on a history of touch inputs.
Salter (US 20200233540 A1) – adjusting user interface regions based on a touch inputs.
Ayoub (US 9,937,416 B2) – adjusting user interface regions based on a touch inputs.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK F RIEGLER whose telephone number is (571)270-3625. The examiner can normally be reached M-F 9:30am-6:00pm, ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kieu Vu can be reached at (571) 272-4057. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK F RIEGLER/ Primary Examiner, Art Unit 2171
Prosecution Insights