Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to application 18927061 filed on October 25, 2024.
Claims 1-20 are pending.
Information Disclosure Statement
As required by M.P.E.P. 609(C), the applicant’s submission of the Information Disclosure Statement dated October 25, 2024 is acknowledged by the examiner and the cited references have been considered in the examination of the claims now pending. As required by M.P.E.P 609, a copy of the PTOL-1449 initialed and dated by the examiner is attached to the office action.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5, 8-9, 12-15, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Sohn et al. (US20140267100A1) in further view of Kim et al. (US20160004380A1).
Regarding claim 1, Sohn teaches a method comprising:
maintaining a store of category data, wherein the category data defines a plurality of … touch gestures (a method of controlling the same to discriminate touch patterns input using different portions of a finger ... a memory configured to store a function corresponding to the touch pattern)([0008] and [0010]; touch gestures (i.e., patterns) are defined and stored in memory);
detecting user interaction with a touch-sensitive display screen (FIG. 6 is a flowchart illustrating a method of controlling the electronic device ... The controller 180 may receive a touch input through the touchscreen 151 (S 101))([0126] and [0127]);
classifying the user interaction according to the category data, including determining that the user interaction corresponds to a first … touch gesture of the plurality of … touch gestures (The controller 180 may sense the touch pattern of the touch input through the sensing unit 140 (S102) ... the sensing unit 140 can discriminate the touch patterns by magnitudes ... based on contact areas between the touch patterns ... etc. … The controller 180 can recognize a touch pattern)([0130], [0131], [0132], and [0140]; touch patterns are recognized); and
controlling an action of the touch sensitive display screen based upon determining that the user interaction corresponds to the first … touch gesture (The controller 180 may execute a function corresponding to the sensed touch pattern (S 103))([0140]).
Sohn differs from the claim in that Sohn does not explicitly teach the gestures are categorized based on size. However, categorizing gestures based on size is taught by Kim (FIG. 1 is a flow chart illustrating a method of recognizing a hand shape for performing a touch action ... data for the region is processed at operation 103 and set of parameters are calculated. Based on the calculated parameters, the shape is identified at operation 104 ...The set of parameters ... comprises one or more of an area of a region ... Referring to FIG. 7A, in order to identify a fist, parameters such as a height to width ratio, left to right length, and a perimeter of a bounding box are used)([0051], [0053], [0054], [0056], and [0116]; gestures (e.g., fist) is categorized based on sizing parameters (e.g., area, height, width, etc.)).
The examiner notes Sohn and Kim teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn to include the categorizing of Kim such that gestures are categorized based on size. One would be motivated to make such a combination to provide the advantage of ensuring a gesture is valid by comparing additional gesture parameters.
Regarding claim 2, Sohn-Kim teach the method of claim 1, wherein the plurality of size-based touch gestures defines a plurality of finger part gestures (Sohn - the sensed touch pattern corresponds to a touch pattern input according to the nails of two fingers ... touch pattern input using a knuckle of a bent finger of the user)([0163] and [0169]; finger part (e.g., nail, knuckle, etc.) gestures are defined).
Regarding claim 3, Sohn-Kim teach the method of claim 1, wherein classifying the user interaction according to the category data comprises: determining a closest match between size data associated with the user interaction and the plurality of size-based touch gestures (Kim - FIG. 11 is a flow chart illustrating a method of shape matching using a calculated parameter and recorded parameters ... At operation 1103, a difference between a value of a perimeter of the touch shape and the recorded perimeter is computed)([0145] and [0150]).
Regarding claim 5, Sohn-Kim teach the method of claim 1, wherein the plurality of size-based touch gestures comprises: a fingertip gesture; a knuckle gesture; and a distal interphalangeal joint-proximal interphalangeal joint gesture (Sohn - The touch patterns of the touch inputs received through the touchscreen 151 may include at least one of a touch with a fingertip … a touch with a nail … a touch with a knuckle of a bent finger)([0124]; a nail gesture includes distal interphalangeal joint-proximal interphalangeal joint movement).
Regarding claim 8, Sohn-Kim teach the method of claim 1, wherein controlling the action of the touch sensitive display screen comprises: opening an application on the display screen, wherein the application is selected based on the first size-based touch gesture (Sohn - As shown in FIG. 8( b), when the user bends a finger and taps the webpage displayed on the touchscreen 151 with a knuckle of a bent finger twice, the controller 180 can execute a memo application)([0150]).
Regarding claim 9, Sohn-Kim teach the method of claim 1, further comprising:
detecting a second user interaction (Sohn - As shown in FIG. 7( a), the user can touch the touchscreen 151 with a nail F4 ... FIG. 7( b) illustrates an operation of selecting an item of the menu window P1. For example, the user can select at least one item from the menu window P1 through a touch input F5 according to a fingertip)([0143] and [0146]; subsequent gestures are detected);
classifying the second user interaction according to the category data, including determining that the second user interaction corresponds to a second size-based touch gesture of the plurality of size-based and touch gestures (Kim - The controller 180 may sense the touch pattern of the touch input through the sensing unit 140 (S102) ... the sensing unit 140 can discriminate the touch patterns by magnitudes ... based on contact areas between the touch patterns ... etc. … The controller 180 can recognize a touch pattern)([0130], [0131], [0132], and [0140]; subsequent gestures are recognized based on size); and
determining that the second user interaction is not associated with a human user corresponding to the user interaction (Kim - Further, at operation 1402, the touch sensitive device detects a validity of the touch)([0173]); and
ignoring the second user interaction (Kim - If the touch is not determined to be valid at operation 1402, then the touch-sensitive device proceeds to wait)([0174]; if touch is not valid it is ignored).
Regarding system claim 12, the claim generally corresponds to method claim 1, and recites similar features in system form; therefore, the claim is rejected under similar rationale.
Regarding claim 13, Sohn-Kim teach the IHS of claim 12, wherein the program instructions to cause the IHS to perform a matching operation between the physical touch and the store of data comprises program instructions to cause the IHS to: classify a width and a height of the physical touch according to the store of data that defines the plurality of size-based touch gestures (Kim - FIG. 1 is a flow chart illustrating a method of recognizing a hand shape for performing a touch action ... data for the region is processed at operation 103 and set of parameters are calculated. Based on the calculated parameters, the shape is identified at operation 104 ...The set of parameters ... comprises one or more of an area of a region ... Referring to FIG. 7A, in order to identify a fist, parameters such as a height to width ratio, left to right length, and a perimeter of a bounding box are used)([0051], [0053], [0054], [0056], and [0116]; gestures (e.g., fist) is categorized based on sizing parameters including width and height).
Regarding claim 14, Sohn-Kim teach the IHS of claim 12, wherein the plurality of size-based touch gestures further includes a second size-based touch gesture, wherein the second size-based touch gesture is defined by the store of data to be larger than the first size-based touch gesture (Kim - FIG. 6 is a flow chart illustrating a method of identifying various parts of a hand and separating joined fingers ... a fist identification is performed at operation 601 ... If the fist is not identified in the touch region in operation 602, then finger identification is performed at operation 604)([0106], [0107], and [0108]; a fist gesture is larger than a finger gesture).
Regarding claim 15, Sohn-Kim teach the IHS of claim 12, wherein the plurality of size-based touch gestures includes a knuckle touch gesture and a thumb palm touch gesture, both of which are defined by the store of data as being different from the first size-based touch gesture (Sohn - The touch patterns of the touch inputs received through the touchscreen 151 may include at least one of a touch with a fingertip … a touch with a nail … a touch with a knuckle of a bent finger)([0124]; each gesture has a different pattern).
Regarding device claim 18, the claim generally corresponds to method claim 1, and recites similar features in device form; therefore, the claim is rejected under similar rationale.
Claims 4 and 10 are rejected under 35 U.S.C. 103 as being unpatentable over Sohn, Kim, and in further view of Wu et al. (US20130201155A1).
Regarding claim 4, Sohn-Kim teach the method as applied above, Sohn-Kim differs from the claim in that Sohn-Kim fails to teach training a machine learning model on different finger parts and storing the trained model. However, training a machine learning model on different finger parts and storing the trained model is taught by Wu (The finger models 116 can be learned and continually updated based on touch input ... For example, if a user touches the touchscreen 102 and the finger or type of finger is incorrectly identified … a model associated with the pinky finger can be adjusted … The model generation module 228 can generate and update models during training modes for the mobile computing device 202 and during normal operation of the mobile computing device 202 … Models are stored in a finger model repository 230, which can store finger models local and/or remote from the mobile computing device 202)([0039] and [0057]).
The examiner notes Sohn, Kim, and Wu teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the training and storing of Wu such that a machine learning model is trained on different finger parts and is stored. One would be motivated to make such a combination to provide the advantage of continually improving and refining gestures.
Regarding claim 10, Sohn-Kim teach the method as applied above, wherein subsequent user interaction is detected (Sohn - As shown in FIG. 7( a), the user can touch the touchscreen 151 with a nail F4 ... FIG. 7( b) illustrates an operation of selecting an item of the menu window P1. For example, the user can select at least one item from the menu window P1 through a touch input F5 according to a fingertip)([0143] and [0146]), classified (Kim - The controller 180 may sense the touch pattern of the touch input through the sensing unit 140 (S102) ... the sensing unit 140 can discriminate the touch patterns by magnitudes ... based on contact areas between the touch patterns ... etc. … The controller 180 can recognize a touch pattern)([0130], [0131], [0132], and [0140]), and determined to be not associated with the user interaction (Kim - Further, at operation 1402, the touch sensitive device detects a validity of the touch)([0173]).
Sohn-Kim differs from the claim in that Sohn-Kim fails to teach associating user interactions with a second (i.e., individual) user profile. However, associating user interactions with individual user profiles is taught by Wu (The finger models 116 can be user specific, in that the finger models for a first user can be different than the finger models for a second user)([0039]).
The examiner notes Sohn, Kim, and Wu teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the associating of Wu such that user interactions are associated with individual user profiles. One would be motivated to make such a combination to provide the advantage of customizing gesture recognition.
Claims 6, 7, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Sohn, Kim, and in further view of Runde et al. (US20140085209A1).
Regarding claim 6, Sohn-Kim teach the method as applied above, Sohn-Kim differs from the claim in that Sohn-Kim fails to teach controlling a line width using the gesture. However, a gesture for controlling line width is taught by Runde (a touch gesture comprising a two-finger pinch/expansion gesture on the digital marking surface 102 may change the stroke width)([0045]).
The examiner notes Sohn, Kim, and Runde teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the controlling Runde such that a line width is controlled using a gesture. One would be motivated to make such a combination to provide the advantage of controlling additional actions using gestures.
Regarding claim 7, Sohn-Kim teach the method as applied above, Sohn-Kim differs from the claim in that Sohn-Kim fails to teach controlling a line color using the gesture. However, a gesture for controlling line color is taught by Runde (a single-finger spiral-shaped touch gesture on the digital marking surface 102 may progress through a color palette selecting a mark color)([0048]).
The examiner notes Sohn, Kim, and Runde teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the controlling Runde such that a line color is controlled using a gesture. One would be motivated to make such a combination to provide the advantage of controlling additional actions using gestures.
Regarding claim 17, Sohn-Kim teach the system as applied above, Sohn-Kim differs from the claim in that Sohn-Kim fails to teach controlling a line color or width using the gesture. However, a gesture for controlling line color or width is taught by Runde (a touch gesture comprising a two-finger pinch/expansion gesture on the digital marking surface 102 may change the stroke width … a single-finger spiral-shaped touch gesture on the digital marking surface 102 may progress through a color palette selecting a mark color)([0045] and [0048]).
The examiner notes Sohn, Kim, and Runde teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the controlling Runde such that a line color or width is controlled using a gesture. One would be motivated to make such a combination to provide the advantage of controlling additional actions using gestures.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Sohn, Kim, and in further view of Villa et al. (US20170300205A1).
Regarding claim 11, Sohn-Kim teach the method as applied above, wherein a subsequent touch gestures correspond to a part of a hand that is not a finger (Sohn - As shown in FIG. 9( a), the user can touch the touchscreen 151 with a fingertip F7 to draw a picture ... Referring to FIG. 9( b), upon reception of a touch input F8 applied by touching the touchscreen 151 with a knuckle of a bent finger, the controller 180 can execute a function)([0010], [0153], and [0155]; a not finger (e.g., knuckle) gesture is detected), wherein a profile of the part of the hand that is not a finger is generated and stored (Sohn - a memory configured to store a function corresponding to the touch pattern)([0010]; a knuckle profile (i.e., pattern) is stored in memory to recognize the gesture), and wherein the gesture includes a width and height (Kim - Referring to FIG. 7A, in order to identify a fist, parameters such as a height to width ratio, left to right length, and a perimeter of a bounding box are used)([0116]).
Sohn-Kim differs from the claim in that Sohn-Kim fails to teach calibrating an application. However, calibrating an application is taught by Villa (FIG. 3 is an example of a scenario of operating the mobile communication device of FIG. 2B with a camera application with one hand where the original UI buttons are still difficult or inconvenient to reach ... The method 700 begins at block 701. At block 705, the processor 205 performs a calibration of the mobile communication device 200 to facilitate ergonomic placement of at least one control element ... At block 720, the processor 205 prompts the user to touch a region of the touchscreen 291 ... At block 735, the processor 205 displays the at least one control element at a location of the touchscreen 291, wherein the location is based on the performed calibration)([0011], [0066], and [0067]).
The examiner notes Sohn, Kim, and Villa teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the calibrating of Villa such that an application is calibrated. One would be motivated to make such a combination to provide the advantage of facilitating ergonomic gestural input.
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Sohn, Kim, and in further view of “Hand Gesture Recognition using ML Algorithms” by Jayashree Baruah (publicly accessible August 26, 2022); hereinafter referred to as Baruah.
Regarding claim 16, Sohn-Kim teach the system as applied above, wherein classifying determines a closest match (Kim - FIG. 11 is a flow chart illustrating a method of shape matching using a calculated parameter and recorded parameters ... At operation 1103, a difference between a value of a perimeter of the touch shape and the recorded perimeter is computed)([0145] and [0150]).
Sohn-Kim differs from the claim in that Sohn-Kim fails to teach using a training set and associated labels of a k-nearest neighbors algorithm. However, recognizing gestures using a training set and associated labels of a k-nearest neighbors algorithm is taught by Baruah (the process of hand gesture recognition includes two components. One is the collection of the training dataset, which is made up of various sorts of hand gestures that are used to train the system ... There are several Machine Learning algorithms to recognize hand gestures ... K Nearest Neighbors ... This algorithm's fundamental principle is to choose the k-nearest neighbors of a particular input from a training database, and then assign those neighbors to the output that received the most votes among those connected to the inputs that were chosen)(pages 2, 3, and 4).
The examiner notes Sohn, Kim, and Baruah teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the recognizing of Baruah such that a gesture is recognized using a training set and associated labels of a k-nearest neighbors algorithm. One would be motivated to make such a combination to provide the advantage of automating recognition of gestures.
Claim 19 is rejected under 35 U.S.C. 103 as being unpatentable over Sohn, Kim, and in further view of Sa et al. (US20130227418A1).
Regarding claim 19, Sohn-Kim teach the device as applied above, Sohn-Kim differs from the claim in that Sohn-Kim fails to teach displaying a setting screen to associate the gesture with an action. However, displaying a setting screen to associate a gesture with an action is taught by Sa (FIG. 1 illustrates an example graphical user interface 100 as part of an example system allowing users to define custom gesture-to-action mappings ... Interface component 106 illustrates an example button that may be selected in order initiate a process for defining a new gesture ... a user interface is presented that includes interface components associated with defining a mapping between a gesture and an action)([0034], [0037], and [0046]).
The examiner notes Sohn, Kim, and Sa teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the displaying of Sa such that a setting screen is displayed to allow a user to associate a gesture and action. One would be motivated to make such a combination to provide the advantage of allowing a user to customize gestures.
Claim 20 is rejected under 35 U.S.C. 103 as being unpatentable over Sohn, Kim, and in further view of Vries et al. (US20240393933A1).
Regarding claim 20, Sohn-Kim teach the device as applied above, wherein subsequent user interaction is detected (Sohn - As shown in FIG. 7( a), the user can touch the touchscreen 151 with a nail F4 ... FIG. 7( b) illustrates an operation of selecting an item of the menu window P1. For example, the user can select at least one item from the menu window P1 through a touch input F5 according to a fingertip)([0143] and [0146]) and input has a width and height (Kim - Referring to FIG. 7A, in order to identify a fist, parameters such as a height to width ratio, left to right length, and a perimeter of a bounding box are used)([0116]).
Sohn-Kim differs from the claim in that Sohn-Kim fails to teach receiving user input enabling accessibility mode, determining a center of mass, and associating a click with the center of mass. However, receiving user input enabling accessibility mode, determining a center of mass, and associating a click with the center of mass is taught by Vries (with reference to operations performed on a device with a touch-sensitive display system 112. In such embodiments, the focus selector is ... a representative point corresponding to a finger or stylus contact (e.g., a centroid of a respective contact or a point associated with a respective contact) ... in accordance with a determination that the first operation of the first application includes initiating a process to enable an accessibility option ... such as touch assistance ... activating an accessibility option (e.g., a respective option that has been selected by the user during a configuration stage, and/or a default accessibility option automatically selected by the operating system))([0192], [0194], and [0303]; a center of mass (i.e., centroid) is used to click (i.e., select) and interact with an interface).
The examiner notes Sohn, Kim, and Vries teach input recognition. As such, it would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to modify Sohn-Kim to include the receiving, determining, and the associating of Vries such that user input enabling accessibility mode is received, a center of mass is determined, and a click is associated with the center of mass. One would be motivated to make such a combination to provide the advantage of assisting users with input.
Conclusion
The prior art made of record on form PTO-892 and not relied upon is considered pertinent to applicant's disclosure. Applicant is required under 37 C.F.R. § 1.111(c) to consider the reference fully when responding to this action. The document cited therein and enumerated below teaches a method and apparatus for controlling a device based on touch input.
US20050052427A1
US20070177804A1
US20080100586A1
US20100306649A1
US20120154447A1
US20130120279A1
US20150160794A1
US20160357281A1
US20170052631A1
US20180095657A1
US20220083211A1
US20230393865A1
US9612689B2
US10642483B2
US11157107B2
CN116027953A
CN117389454B
PalmTouch: Using the Palm as an Additional Input Modality on Commodity Smartphones
Detecting the size of a users finger on touch
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Yongjia Pan whose telephone number is (571)270-1177. The examiner can normally be reached Monday - Friday, 9:00 AM - 5:00 PM EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Scott Baderman can be reached at 571-272-3644. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YONGJIA PAN/Primary Examiner, Art Unit 2118
Prosecution Insights