DETAILED ACTION
This Office Action is in response to Applicant's Response filed on 01/20/2026 for the above identified application.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 01/20/2026 has been entered.
Claims 1, 9, and 20 are amended. Claims 1-11, 15-17, 19-20, 22, and 25-27 are pending in the application.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-2, 9-10, 17, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Schwesig et al. (US 2019/0011989 A1 hereinafter Schwesig) in view of Yuan (US 2019/0303219 A1), further in view of Kim et al. (US 2008/0089587 A1 hereinafter Kim).
Regarding Claim 1, Schwesig teaches an electronic device ([0041] Computing device 102), comprising:
one or more processors ([0041] Computing device 102 includes a processing system 302 (e.g., processors, CPUs, GPUs));
one or more memories coupled to the one or more processors and configured to store ([0041] Computing device 102 includes a processing system 302 (e.g., processors, CPUs, GPUs) and computer-readable storage media 304):
a plurality of applications, wherein the plurality of applications comprises a system application and a third-party application ([0041] Computing device 102 includes a processing system 302 (e.g., processors, CPUs, GPUs) and computer-readable storage media 304. Applications 206 (e.g., media consumption, browsers, third-party applications, or an and/or an operating system) (i.e., comprising a system application and a third-party application) are embodied as computer-readable instructions on the computer-readable storage media 302 and are executed by the processing system 302 to perform operations), and wherein the third-party application does not define a correspondence between a mid-air gesture made by a user and an application operation instruction ([0029] the 3D object detection system 120 may detect an object and movement of the object; data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture; result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth; the applications 206 cause operations to be performed; [0030] detection of the gesture by the gesture component 104 may be abstracted away from the applications 206 such that the applications 206 are not aware of how the gesture is detected, but may leverage detection of the gesture to control operations of the computing device 102); and
one or more computer programs ([0041] Applications 206 (e.g., media consumption, browsers, third-party applications, or an and/or an operating system) are embodied as computer-readable instructions on the computer-readable storage media 302 and are executed by the processing system 302 to perform operations);
a display coupled to the one or more processors and configured to display an interface of the application ([0090] Computing device 800 also includes audio and/or video processing system 816 that processes audio data and/or passes through the audio and video data to audio system 818 and/or to display system 820 (e.g., a screen of a smart phone or camera); [0035]user interface (UI) navigation gestures 220 may be defined to control operations involving navigation within a user interface; this may include pinch gestures to zoom a display of digital content, pan gestures to navigate directionally, scroll gestures, and so forth); and
a detection component configured to detect the mid-air gesture, wherein the mid-air gesture is not in contact with the electronic device, wherein the detection component comprises any one of a camera, an infrared sensor disposed below the display, an ultrasonic sensor disposed below the display, or an electromagnetic induction component disposed below the display ([0029] the 3D object detection system 120 may detect an object and movement of the object. Data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; [0017] a gesture component incorporates techniques usable to detect gestures, such as a touchpad, touchscreen functionality, cameras; a three-dimensional object detection system may be employed in which gestures are detected in free space, without any attachment or peripheral device connected to a corresponding body part, such as a hand; the three-dimensional object detection system may leverage transmission and reception of radio signals to detect orientation and movement of an object (e.g., a body part of a user such as a hand) in three-dimensional space (i.e., not in contact with the electronic device); [0027] the detection may be performed using a variety of different techniques, such as cameras, sound waves),
wherein the one or more computer programs comprise instructions that, when executed by the one or more processors, cause the electronic device ([0041] Computing device 102 includes a processing system 302 and computer-readable storage media; applications (e.g., media consumption, browsers, third-party applications, or an and/or an operating system) are embodied as computer-readable instructions on the computer-readable storage media and are executed by the processing system to perform operations) to:
display, on the display, the interface after the application is opened ([0090] Computing device 800 also includes audio and/or video processing system 816 that processes audio data and/or passes through the audio and video data to audio system 818 and/or to display system 820 (e.g., a screen of a smart phone or camera); [0035] user interface (UI) navigation gestures 220 may be defined to control operations involving navigation within a user interface; this may include pinch gestures to zoom a display of digital content, pan gestures to navigate directionally, scroll gestures, and so forth (i.e., interface of application)) ;
generate a general-purpose input event based on the mid-air gesture ([0029] the 3D object detection system 120 may detect an object and movement of the object. Data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture. A result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth. The applications 206, for instance, may cause operations to be performed that involve input/output device 208, such as to render digital content, transform data, and so forth; [0032] The gesture module 122 accepts data as an input from the 3D object detection system 120 that describes this positioning and movement. This data is then compared with a gesture library 202 to recognize the gesture 210, an operation to be performed that corresponds to the gesture, and an amount to which this operation is to be performed; [0034] this functionality may be shared with a plurality of applications 206 as well as across a variety of types of devices and the user 124 is provided with a consistent collection of gestures that perform as expected across these devices - thus, generating general purpose input event); and
perform, by the third-party application, the application operation instruction based on the general-purpose input event when the detection component detects the mid-air gesture ([0029] the 3D object detection system 120 may detect an object and movement of the object; data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture; result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth; the applications 206 cause operations to be performed (i.e., performing by the third-party application, application operation instruction based on the general-purpose input event)).
Schwesig does not explicitly disclose wherein display an interface of the third-party application and display the interface after the third-party application is opened. However, Schwesig teaches that Computing device 800 also includes audio and/or video processing system 816 that processes audio data and/or passes through the audio and video data to audio system 818 and/or to display system 820 (e.g., a screen of a smart phone or camera); user interface (UI) navigation gestures 220 may be defined to control operations involving navigation within a user interface; this may include pinch gestures to zoom a display of digital content, pan gestures to navigate directionally, scroll gestures, and so forth; the 3D object detection system 120 may detect an object and movement of the object; data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture; result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth; the applications 206 cause operations to be performed (see [0029], [0035], [0090]), which implies that the gestures would cause corresponding operations to be performed on the displayed third-party application interface.
In a similar field of endeavor, Yuan teaches wherein display an interface of the third-party application and display the interface after the third-party application is opened ([0042] second interface is developed by a third party service provider (e.g., third party platform 104); third party platform 104 may develop a corresponding second interface in the style of and/or includes interactive elements that are with its corresponding application; [0050] as shown in display screen 204, the second interface displays button B7; the user can select button B7 to request service(s) associated with the second application that are provided by a third party service provider (e.g., server) that had generated/provided the interface data from which the second interface was generated; in response to the user selection of button B7, an operation request is sent from the terminal to the third party server; then the third party service server, responding to the operation request, may generate corresponding operation response information - thus, displaying the interface of a third-party application after it is opened).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein display an interface of the third-party application and display the interface after the third-party application is opened, as taught by Yuan into Schwesig. Doing so would be desirable because it would provide a convenient or a fast way to provide a service feature (Yuan [0036]).
However, Schwesig and Yuan do not expressly teach wherein display, on the display, an operation mark that relates to the mid-air gesture, wherein the operation mark simulates an actual touch operation performed by the user on a touchscreen to prompt the user with the general-purpose input event to which the electronic device responds, and wherein the operation mark is independent of the third- party application.
In the same field of endeavor, Kim teaches wherein display, on the display, an operation mark that relates to the mid-air gesture ([0053] the pointer activation command can be represented by a hand gesture image in which joints of fingers are bent except for the thumb and index fingers (i.e., mid-air gesture); if this hand gesture is input through the camera, the pointer moves on the screen of the display (i.e., operation mark that relates to the mid-air gesture/ hand gesture in which joints of fingers are bent except for the thumb and index fingers)), wherein the operation mark simulates an actual touch operation performed by the user on a touchscreen to prompt the user with the general-purpose input event to which the electronic device responds ([0090] the mobile phone can be implemented with a hand gesture recognition input mode together with a keypad input mode and touchpad input mode such that the user can select any of the three input modes; [0046] the display has a screen for displaying various menus for the application programs and information input or requested by a user; when the display LCD implemented with a touchscreen, the display can be used as an additional input means (i.e., touch operation on touchscreen as input means); [0053] the pointer activation command can be represented by a hand gesture image in which joints of fingers are bent except for the thumb and index fingers; if this hand gesture is input through the camera, the pointer moves on the screen of the display; if all the fingers are straight back, the movement of the pointer stops - thus, the operation mark/ pointer moving or stopping on the screen simulates the touch operation of user fingers on touchscreen and prompts the user that hand gesture in which joints of fingers are bent except for the thumb and index fingers or all the fingers straight back/ general-purpose input event is being responded to by the electronic device), and wherein the operation mark is independent of the third- party application ([0077] the controller establishes mappings between the first to nth control images and a first to nth control commands in the lookup table; the control commands include system operation-related commands and application control commands; [0038] the hand gesture recognition input system and method of the present invention is described in association with an operation of a pointer on the screen and can operate other applications installed in the mobile phones; [0053] the pointer activation command can be represented by a hand gesture image in which joints of fingers are bent except for the thumb and index fingers; if this hand gesture is input through the camera, the pointer moves on the screen of the display; if all the fingers are straight back, the movement of the pointer stops - thus, the operation mark/ pointer moving or stopping on the screen is independent of the third- party application).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein display, on the display, an operation mark that relates to the mid-air gesture, wherein the operation mark simulates an actual touch operation performed by the user on a touchscreen to prompt the user with the general-purpose input event to which the electronic device responds, and wherein the operation mark is independent of the third- party application, as taught by Kim into Schwesig and Yuan. Doing so would be desirable because it would provide a novel input technology for a space-limited mobile phone that is capable of facilitating information conveyance as well as mobile phone control (Kim [0009]).
As to dependent Claim 2, Schwesig, Yuan, and Kim teach all the limitations of claim 1. Schwesig further teaches wherein the mid-air gesture is a wave of a hand of the user or a translational move of the hand, and wherein the application operation instruction comprises fast-forwarding a first multimedia file of the third- party application, rewinding the first multimedia file, turning play volume up, turning the play volume down, turning display brightness up, turning the display brightness down, or switching the first multimedia file to another multimedia file ([0019] The gesture may be recognized by the gesture component to increase volume; the gesture component then expose via the application programming interfaces to an application to increase the volume in rendering of digital content, e.g., audio or multimedia content; [0029] the 3D object detection system 120 may detect an object and movement of the object; data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture; result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth (i.e., including third-party applications); the applications 206 cause operations to be performed; [0031] gesture performed by a hand of the user; a hand of the user that mimics the turning of a physical knob (equivalent to wave of a hand of the user or translational move of the hand); [0032] the gesture module 122 accepts data as an input from the 3D object detection system 120 that describes this positioning and movement; this data is then compared with a gesture library 202 to recognize the gesture 210, an operation to be performed that corresponds to the gesture, and an amount to which this operation is to be performed; the gesture 210 is a media gesture 218 that is configured to adjust a level of volume for audio output; the level of volume is raised or lowered by rotating to the left 214 or right 216, respectively).
Regarding Claim 9, Schwesig teaches an electronic device ([0041] Computing device 102), comprising:
one or more processors ([0041] Computing device 102 includes a processing system 302 (e.g., processors, CPUs, GPUs));
one or more memories coupled to the one or more processors and configured to store ([0041] Computing device 102 includes a processing system 302 (e.g., processors, CPUs, GPUs) and computer-readable storage media 304):
a plurality of applications comprising a system application and a third-party application ([0041] Computing device 102 includes a processing system 302 (e.g., processors, CPUs, GPUs) and computer-readable storage media 304. Applications 206 (e.g., media consumption, browsers, third-party applications, or an and/or an operating system) (i.e., comprising a system application and a third-party application) are embodied as computer-readable instructions on the computer-readable storage media 302 and are executed by the processing system 302 to perform operations), wherein the third-party application does not include a correspondence between a mid-air gesture made by a user and an application operation instruction ([0029] the 3D object detection system 120 may detect an object and movement of the object; data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture; result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth; the applications 206 cause operations to be performed; [0030] detection of the gesture by the gesture component 104 may be abstracted away from the applications 206 such that the applications 206 are not aware of how the gesture is detected, but may leverage detection of the gesture to control operations of the computing device 102); and
one or more computer programs ([0041] Applications 206 (e.g., media consumption, browsers, third-party applications, or an and/or an operating system) are embodied as computer-readable instructions on the computer-readable storage media 302 and are executed by the processing system 302 to perform operations);
a display coupled to the one or more processors and configured to display a first interface of the application ([0090] Computing device 800 also includes audio and/or video processing system 816 that processes audio data and/or passes through the audio and video data to audio system 818 and/or to display system 820 (e.g., a screen of a smart phone or camera); [0035]user interface (UI) navigation gestures 220 may be defined to control operations involving navigation within a user interface; this may include pinch gestures to zoom a display of digital content, pan gestures to navigate directionally, scroll gestures, and so forth); and
a detection component comprising any one of a camera, an infrared sensor disposed below the display, an ultrasonic sensor disposed below the display, or an electromagnetic induction component disposed below the display ([0029] the 3D object detection system 120 may detect an object and movement of the object. Data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; [0017] a gesture component incorporates techniques usable to detect gestures, such as a touchpad, touchscreen functionality, cameras; a three-dimensional object detection system may be employed in which gestures are detected in free space, without any attachment or peripheral device connected to a corresponding body part, such as a hand; the three-dimensional object detection system may leverage transmission and reception of radio signals to detect orientation and movement of an object (e.g., a body part of a user such as a hand) in three-dimensional; [0027] the detection may be performed using a variety of different techniques, such as cameras, sound waves),
wherein the one or more computer programs comprise instructions that, when executed by the one or more processors, cause the electronic device ([0041] Computing device 102 includes a processing system 302 and computer-readable storage media; applications (e.g., media consumption, browsers, third-party applications, or an and/or an operating system) are embodied as computer-readable instructions on the computer-readable storage media and are executed by the processing system to perform operations) to:
display, on the display, the first interface after the application is opened ([0090] Computing device 800 also includes audio and/or video processing system 816 that processes audio data and/or passes through the audio and video data to audio system 818 and/or to display system 820 (e.g., a screen of a smart phone or camera); [0035]user interface (UI) navigation gestures 220 may be defined to control operations involving navigation within a user interface; this may include pinch gestures to zoom a display of digital content, pan gestures to navigate directionally, scroll gestures, and so forth (i.e., interface of application));
generate a general-purpose input event based on the mid-air gesture ([0029] the 3D object detection system 120 may detect an object and movement of the object. Data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture. A result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth. The applications 206, for instance, may cause operations to be performed that involve input/output device 208, such as to render digital content, transform data, and so forth; [0032] The gesture module 122 accepts data as an input from the 3D object detection system 120 that describes this positioning and movement. This data is then compared with a gesture library 202 to recognize the gesture 210, an operation to be performed that corresponds to the gesture, and an amount to which this operation is to be performed; [0034] this functionality may be shared with a plurality of applications 206 as well as across a variety of types of devices and the user 124 is provided with a consistent collection of gestures that perform as expected across these devices - thus, generating general purpose input event); and
perform, by the third-part application, the application operation instruction based on the general-purpose input event when the detection component detects the mid-air gesture ([0029] the 3D object detection system 120 may detect an object and movement of the object; data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture; result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth; the applications 206 cause operations to be performed (i.e., performing by the third-party application, application operation instruction based on the general-purpose input event)).
Schwesig does not explicitly disclose wherein display a first interface of the third-party application and display the first interface after the third-party application is opened. However, Schwesig teaches that Computing device 800 also includes audio and/or video processing system 816 that processes audio data and/or passes through the audio and video data to audio system 818 and/or to display system 820 (e.g., a screen of a smart phone or camera); user interface (UI) navigation gestures 220 may be defined to control operations involving navigation within a user interface; this may include pinch gestures to zoom a display of digital content, pan gestures to navigate directionally, scroll gestures, and so forth; the 3D object detection system 120 may detect an object and movement of the object; data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; the gesture module 122 also identifies an operation to be performed that corresponds to the gesture; result of this is then output by the gesture module 122 via one or more application programming interfaces 204 to applications 206 that are executable by the computing device 102, such as third-party applications, an operating system, and so forth; the applications 206 cause operations to be performed (see [0029], [0035], [0090]), which implies that the gestures would cause corresponding operations to be performed on the displayed third-party application interface.
In a similar field of endeavor, Yuan teaches wherein display a first interface of the third-party application and display the first interface after the third-party application is opened ([0042] second interface is developed by a third party service provider (e.g., third party platform 104); third party platform 104 may develop a corresponding second interface in the style of and/or includes interactive elements that are with its corresponding application; [0050] as shown in display screen 204, the second interface displays button B7; the user can select button B7 to request service(s) associated with the second application that are provided by a third party service provider (e.g., server) that had generated/provided the interface data from which the second interface was generated; in response to the user selection of button B7, an operation request is sent from the terminal to the third party server; then the third party service server, responding to the operation request, may generate corresponding operation response information - thus, displaying the first interface of a third-party application after it is opened).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein display a first interface of the third-party application and display the first interface after the third-party application is opened, as taught by Yuan into Schwesig. Doing so would be desirable because it would provide a convenient or a fast way to provide a service feature (Yuan [0036]).
However, Schwesig and Yuan do not expressly teach wherein display, on the display, an operation mark that relates to the mid-air gesture, wherein the operation mark simulates an actual touch operation performed by the user on a touchscreen to prompt the user with the general-purpose input event to which the electronic device responds, and wherein the operation mark is independent of the third-party application.
In the same field of endeavor, Kim teaches wherein display, on the display, an operation mark that relates to the mid-air gesture ([0053] the pointer activation command can be represented by a hand gesture image in which joints of fingers are bent except for the thumb and index fingers (i.e., mid-air gesture); if this hand gesture is input through the camera, the pointer moves on the screen of the display (i.e., operation mark that relates to the mid-air gesture/ hand gesture in which joints of fingers are bent except for the thumb and index fingers)), wherein the operation mark simulates an actual touch operation performed by the user on a touchscreen to prompt the user with the general-purpose input event to which the electronic device responds ([0090] the mobile phone can be implemented with a hand gesture recognition input mode together with a keypad input mode and touchpad input mode such that the user can select any of the three input modes; [0046] the display has a screen for displaying various menus for the application programs and information input or requested by a user; when the display LCD implemented with a touchscreen, the display can be used as an additional input means (i.e., touch operation on touchscreen as input means); [0053] the pointer activation command can be represented by a hand gesture image in which joints of fingers are bent except for the thumb and index fingers; if this hand gesture is input through the camera, the pointer moves on the screen of the display; if all the fingers are straight back, the movement of the pointer stops - thus, the operation mark/ pointer moving or stopping on the screen simulates the touch operation of user fingers on touchscreen and prompts the user that hand gesture in which joints of fingers are bent except for the thumb and index fingers or all the fingers straight back/ general-purpose input event is being responded to by the electronic device), and wherein the operation mark is independent of the third-party application ([0077] the controller establishes mappings between the first to nth control images and a first to nth control commands in the lookup table; the control commands include system operation-related commands and application control commands; [0038] the hand gesture recognition input system and method of the present invention is described in association with an operation of a pointer on the screen and can operate other applications installed in the mobile phones; [0053] the pointer activation command can be represented by a hand gesture image in which joints of fingers are bent except for the thumb and index fingers; if this hand gesture is input through the camera, the pointer moves on the screen of the display; if all the fingers are straight back, the movement of the pointer stops - thus, the operation mark/ pointer moving or stopping on the screen is independent of the third- party application).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein display, on the display, an operation mark that relates to the mid-air gesture, wherein the operation mark simulates an actual touch operation performed by the user on a touchscreen to prompt the user with the general-purpose input event to which the electronic device responds, and wherein the operation mark is independent of the third-party application, as taught by Kim into Schwesig and Yuan. Doing so would be desirable because it would provide a novel input technology for a space-limited mobile phone that is capable of facilitating information conveyance as well as mobile phone control (Kim [0009]).
As to dependent Claim 10, Schwesig, Yuan, and Kim teach all the limitations of claim 9. Schwesig further teaches wherein the application operation instruction comprises upward scrolling a page when the mid-air gesture is an upward translational move of a hand of the user or an upward wave of the hand, and wherein the application operation instruction comprises downward scrolling the page when the mid-air gesture is a downward translational move of the hand or a downward wave of the hand ([0021] navigation gestures are defined to control navigation-relation operations of an application, such as to pan, zoom, scroll, grab-and-drop, and so forth; [0035] navigation gestures 220 may be defined to control operations involving navigation within a user interface; this include pinch gestures to zoom a display of digital content, pan gestures to navigate directionally, scroll gestures, and so forth; [0026] the gesture module 122 may cause an operation to be performed, such as to detect a rightward swipe by a user's hand and cause a user interface output by the computing device 102 to move a corresponding direction).
As to dependent Claim 17, Schwesig, Yuan, and Kim teach all the limitations of claim 1. Schwesig further teaches wherein the general- purpose input event is a sliding event when the mid-air gesture is a wave of a hand of the user or a translational move of the hand ([0021] navigation gestures are defined to control navigation-relation operations of an application, such as to pan, zoom, scroll, grab-and-drop, and so forth; [0026] the gesture module 122 may cause an operation to be performed, such as to detect a rightward swipe by a user's hand and cause a user interface output by the computing device 102 to move a corresponding direction; [0035] navigation gestures 220 may be defined to control operations involving navigation within a user interface; this include pinch gestures to zoom a display of digital content, pan gestures to navigate directionally, scroll gestures, and so forth), and wherein the general-purpose input event is a tapping event when the mid-air gesture is a move of a palm ([0037] a micro-gesture performed by all or a portion of the hand (e.g., a single-tap gesture, a double-tap gesture, a left-swipe, a forward-swipe, a right-swipe, a finger making a shape, etc.). Kim further teaches wherein tapping event when the mid-air gesture is a move toward the electronic device ([0054] the layer navigation command is represented by a gesture moving a fist alternately towards and away from the camera).
Claim 20 is a method claim similar to the device claim 1 above and therefore, rejected for the same reasons.
Claims 4, 7-8, 11, 19, 22, 25, and 27 are rejected under 35 U.S.C. 103 as being unpatentable over Schwesig in view of Yuan and Kim, further in view of Weber et al. (US 9,377,860 B1 hereinafter Weber).
As to dependent Claim 4, Schwesig, Yuan, and Kim teach all the limitations of claim 2. Schwesig further teaches wherein the plurality of applications include an audio player ([0019] The gesture may be recognized by the gesture component to increase volume; the gesture component then expose via the application programming interfaces to an application to increase the volume in rendering of digital content, e.g., audio or multimedia content; [0034] media gestures 218 may be defined as a gesture library 202 to control operations involving creation and/or rendering of media, such as digital images, digital audio, video, multimedia, and other types of digital content rendered by the computing device).
However, Schwesig, Yuan, and Kim do not expressly teach wherein the application operation instruction comprises fast-forwarding the first multimedia file when the mid- air gesture is a rightward translational move of the hand or a rightward wave of the hand and wherein the application operation instruction comprises rewinding the first multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand.
In the same field of endeavor, Weber teaches wherein the application operation instruction comprises fast-forwarding the first multimedia file when the mid- air gesture is a rightward translational move of the hand or a rightward wave of the hand (column 7, lines 10 to 32 - the computing device determines the clockwise motion of the driver's hand based at least in part on data received from one or more sensors of the computing device; the computing device monitor the gesture area, and detect when the driver's hand comes within the gesture area; when the device detects the user's hand, the device can capture images of the driver's hand as the driver performs the motion; the device can then analyze the images to detect the clockwise motion (equivalent to rightward translational move of the hand or a rightward wave of the hand), and in response detecting the motion, the device can cause the media track to fast-forward), and wherein the application operation instruction comprises rewinding the first multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand (column 3, lines 13 to 20 - while the track is playing, when the user's hand moves in the gesture area in a counter-clockwise circular motion (equivalent to leftward translational move of a hand or a leftward wave of a hand), the device can cause the point of play to reverse backward in the time sequence of the current track).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the application operation instruction comprises fast-forwarding the first multimedia file when the mid- air gesture is a rightward translational move of the hand or a rightward wave of the hand and wherein the application operation instruction comprises rewinding the first multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand, as taught by Weber into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow to operate the device where touch input is not possible or permitted, or where diverting their visual focus to the device is dangerous (such as when driving) (Weber, column 1, lines 20 to 24).
As to dependent Claim 7, Schwesig, Yuan, and Kim teach all the limitations of claim 1. However, Schwesig, Yuan, and Kim do not expressly teach wherein the mid-air gesture is a move of a palm toward the electronic device, wherein the application operation instruction comprises playing a first multimedia file of the third-party application, pausing playing of the first multimedia file, or switching the first multimedia file to another multimedia file.
In the same field of endeavor, Weber teaches wherein the mid-air gesture is a move of a palm toward the electronic device, wherein the application operation instruction comprises playing a first multimedia file of the third-party application, pausing playing of the first multimedia file, or switching the first multimedia file to another multimedia file (column 7, lines 31 to 38 - the driver's hand is moving in a forward motion in the gesture area relative to the device to cause a media track currently playing to pause).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the mid-air gesture is a move of a palm toward the electronic device, wherein the application operation instruction comprises playing a first multimedia file of the third-party application, pausing playing of the first multimedia file, or switching the first multimedia file to another multimedia file, as taught by Weber into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow to operate the device where touch input is not possible or permitted, or where diverting their visual focus to the device is dangerous (such as when driving) (Weber, column 1, lines 20 to 24).
As to dependent Claim 8, Schwesig, Yuan, Kim, and Weber teach all the limitations of claim 7. Weber further teaches wherein the plurality of applications include a first video player (column 4, lines 1 to 5 - a media player operating on the computing device; column 2, lines 55 to 65 - “media player” refers to as a software application used by users to listen to media; “media” refers to a song, video), wherein the interface comprises a playing control and a switching control (See fig. 3B for playing control and fig. 2B for switching control), wherein performing the application operation instruction comprises performing the application operation instruction based on the mid-air gesture, a location at which the mid-air gesture occurs, and a current status of the first video player (column 7, lines 1 to 32 - media content being presented via the device (i.e., status); the computing device determines the clockwise motion of the driver's hand based at least in part on data received from one or more sensors of the computing device; the computing device monitor the gesture area (i.e., location), and detect when the driver's hand comes within the gesture area; when the device detects the user's hand, the device can capture images of the driver's hand as the driver performs the motion; the device can then analyze the images to detect the clockwise motion, and in response detecting the motion, the device can cause the media track to fast-forward (i.e., response based on gesture)), wherein the application operation instruction comprises pausing playing of the first multimedia file when the location at which the mid-air gesture occurs is close to the playing control and the first multimedia file is currently being played (column 7, lines 31 to 42 - the driver's hand is moving in a forward motion in the gesture area relative to the device to cause a media track currently playing to pause; while the track is playing, moving a hand towards the device within the gesture area with the palm open and facing downward will pause the track at the moment directly after the gesture is executed. See fig. 3B for playing control), wherein the application operation instruction comprises playing the first multimedia file when the location at which the mid-air gesture occurs is close to the playing control and the first multimedia file is not currently played (column 7, lines 62 to 68 - the user can make a subsequent forward motion of their hand towards the device within the gesture area to cause the device to resume play of the media track; the track will resume play where it was left off from the beginning of the paused state at the moment directly after the gesture is executed. See fig. 3B for playing control), and wherein the application operation instruction comprises switching the first multimedia file to the another multimedia file when the location at which the mid-air gesture occurs is close to the switching control (column 6, lines 38 to 55 - the user moves their hand in a left-to-right motion relative to the mobile device, such as in a gesture area of the mobile device; as shown in FIG. 2B, the left-to-right motion can be interpreted by the mobile device to cause the mobile device to skip to the next track (as indicated by the graphical indication 240 on the mobile device). See fig. 2B for switching control).
As to dependent Claim 11, Schwesig, Yuan, and Kim teach all the limitations of claim 9. However, Schwesig, Yuan, and Kim do not expressly teach wherein the application operation instruction comprises switching the first interface to a second interface when the mid-air gesture is a rightward translational move of a hand of the user or a rightward wave of the hand, wherein the application operation instruction comprises switching the first interface to a third interface when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand, and wherein either: the first interface corresponds to a first function of the one or more applications, the second interface corresponds to a second function of the one or more applications, and the third interface corresponds to a third function of the one or more applications; or the one or more applications include a reader, and the first interface, the second interface, and the third interface are respectively three pages displayed by the reader.
In the same field of endeavor, Weber teaches wherein the application operation instruction comprises switching the first interface to a second interface when the mid-air gesture is a rightward translational move of a hand of the user or a rightward wave of the hand (column 2, lines 55 to 60 - “track”, “media track”, “media” or “content” refers to a single song, video, or piece of sequential content with a finite duration; column 3, lines 39 to 43 - the user can perform a gesture to skip to a next track; while the track is playing, when the user's hand moves in a left-to-right motion, the device can cause the point of play to move to the beginning of the next track within the media - thus, switching the first interface/ current track to the next track/ second interface when the mid-air gesture is a rightward translational move/ wave of a hand), wherein the application operation instruction comprises switching the first interface to a third interface when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand (column 3, lines 48 to 50 - when the user's hand moves in a right-to-left motion (i.e., mid-air gesture is a leftward translational move/ wave of a hand) in the gesture area, the device can cause the point of play to move to the beginning of the previous track (i.e., third interface)), and wherein either: the first interface corresponds to a first function of the one or more applications, the second interface corresponds to a second function of the one or more applications, and the third interface corresponds to a third function of the one or more applications; or the one or more applications include a reader, and the first interface, the second interface, and the third interface are respectively three pages displayed by the reader (column 2, lines 55 to 65 - “media player” refers to as a software application used by users to listen to media; “media” refers to a song, video; “track”, “media track”, “media” or “content” refers to a single song, video, or piece of sequential content with a finite duration; column 3, lines 39 to 43 - the user can perform a gesture to skip to a next track; while the track is playing (i.e., first interface/ first function of the media application), when the user's hand moves in a left-to-right motion, the device can cause the point of play to move to the beginning of the next track (i.e., second interface/ second function of the media application) within the media; column 3, lines 48 to 50 - when the user's hand moves in a right-to-left motion in the gesture area, the device can cause the point of play to move to the beginning of the previous track (i.e., third interface / third function of the media application).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the application operation instruction comprises switching the first interface to a second interface when the mid-air gesture is a rightward translational move of a hand of the user or a rightward wave of the hand, wherein the application operation instruction comprises switching the first interface to a third interface when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand, and wherein either: the first interface corresponds to a first function of the one or more applications, the second interface corresponds to a second function of the one or more applications, and the third interface corresponds to a third function of the one or more applications; or the one or more applications include a reader, and the first interface, the second interface, and the third interface are respectively three pages displayed by the reader, as taught by Weber into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow to operate the device where touch input is not possible or permitted, or where diverting their visual focus to the device is dangerous (such as when driving) (Weber, column 1, lines 20 to 24).
As to dependent Claim 19, Schwesig, Yuan, and Kim teach all the limitations of claim 2. However, Schwesig, Yuan, and Kim do not expressly teach wherein the application operation instruction comprises fast-forwarding the first multimedia file when the mid-air gesture is a rightward translational move of the hand or a rightward wave of the hand, and wherein the response operation application operation instruction comprises rewinding the first multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand.
In the same field of endeavor, Weber teaches wherein the application operation instruction comprises fast-forwarding the first multimedia file when the mid-air gesture is a rightward translational move of the hand or a rightward wave of the hand (column 7, lines 10 to 32 - the computing device determines the clockwise motion of the driver's hand based at least in part on data received from one or more sensors of the computing device; the computing device monitor the gesture area, and detect when the driver's hand comes within the gesture area; when the device detects the user's hand, the device can capture images of the driver's hand as the driver performs the motion; the device can then analyze the images to detect the clockwise motion, and in response detecting the motion, the device can cause the media track to fast-forward), and wherein the response operation application operation instruction comprises rewinding the first multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand (column 3, lines 13 to 20 - while the track is playing, when the user's hand moves in the gesture area in a counter-clockwise circular motion, the device can cause the point of play to reverse backward in the time sequence of the current track).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the application operation instruction comprises fast-forwarding the first multimedia file when the mid-air gesture is a rightward translational move of the hand or a rightward wave of the hand, and wherein the response operation application operation instruction comprises rewinding the first multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand, as taught by Weber into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow to operate the device where touch input is not possible or permitted, or where diverting their visual focus to the device is dangerous (such as when driving) (Weber, column 1, lines 20 to 24).
As to dependent Claim 22, Schwesig, Yuan, and Kim teach all the limitations of claim 1. However, Schwesig, Yuan, and Kim do not expressly teach wherein the application operation instruction comprises playing a first multimedia file of the third-party application, pausing playing of the first multimedia file, adjusting play progress of the first multimedia file, adjusting play volume, adjusting display brightness, or switching the first multimedia file to another multimedia file.
In the same field of endeavor, Weber teaches wherein the application operation instruction comprises playing a first multimedia file of the third-party application, pausing playing of the first multimedia file, adjusting play progress of the first multimedia file, adjusting play volume, adjusting display brightness, or switching the first multimedia file to another multimedia file (column 7, lines 10 to 32 - the computing device determines the clockwise motion of the driver's hand based at least in part on data received from one or more sensors of the computing device; the computing device monitor the gesture area, and detect when the driver's hand comes within the gesture area; when the device detects the user's hand, the device can capture images of the driver's hand as the driver performs the motion; the device can then analyze the images to detect the clockwise motion, and in response detecting the motion, the device can cause the media track to fast-forward (i.e., adjusting play progress of the multimedia file); using gesture to pause a media track playing on the device).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the application operation instruction comprises playing a first multimedia file of the third-party application, pausing playing of the first multimedia file, adjusting play progress of the first multimedia file, adjusting play volume, adjusting display brightness, or switching the first multimedia file to another multimedia file, as taught by Weber into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow to operate the device where touch input is not possible or permitted, or where diverting their visual focus to the device is dangerous (such as when driving) (Weber, column 1, lines 20 to 24).
As to dependent Claim 25, Schwesig, Yuan, and Kim teach all the limitations of claim 9. However, Schwesig, Yuan, and Kim do not expressly teach wherein the mid-air gesture is a wave of a hand of the user or a translational move of the hand, and wherein the application operation instruction comprises fast-forwarding a first multimedia file of the third-party application, rewinding the first multimedia file, turning play volume up, turning the play volume down, turning display brightness up, turning the display brightness down, or switching the first multimedia file to another multimedia file.
In the same field of endeavor, Weber teaches wherein the mid-air gesture is a wave of a hand of the user or a translational move of the hand, and wherein the application operation instruction comprises fast-forwarding a first multimedia file of the third-party application, rewinding the first multimedia file, turning play volume up, turning the play volume down, turning display brightness up, turning the display brightness down, or switching the first multimedia file to another multimedia file (column 7, lines 10 to 36 - the computing device determines the clockwise motion of the driver's hand; the device can then analyze the images to detect the clockwise motion, and in response detecting the motion, the device can cause the media track to fast-forward; using gesture to pause a media track playing on the device when driver's hand is moving in a forward motion).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the mid-air gesture is a wave of a hand of the user or a translational move of the hand, and wherein the application operation instruction comprises fast-forwarding a first multimedia file of the third-party application, rewinding the first multimedia file, turning play volume up, turning the play volume down, turning display brightness up, turning the display brightness down, or switching the first multimedia file to another multimedia file, as taught by Weber into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow to operate the device where touch input is not possible or permitted, or where diverting their visual focus to the device is dangerous (such as when driving) (Weber, column 1, lines 20 to 24).
As to dependent Claim 27, Schwesig, Yuan, Kim, and Weber teach all the limitations of claim 25. Weber further teaches wherein the plurality of applications include an audio player (column 4, lines 1 to 5 - a media player operating on the computing device; column 2, lines 55 to 65 - “media player” refers to as a software application used by users to listen to media; “media” refers to a song, video), wherein the application operation instruction comprises fast-forwarding the first multimedia file when the mid-air gesture is a rightward translational move of the hand or a rightward wave of the hand (column 7, lines 10 to 32 - the computing device determines the clockwise motion of the driver's hand based at least in part on data received from one or more sensors of the computing device; the computing device monitor the gesture area, and detect when the driver's hand comes within the gesture area; when the device detects the user's hand, the device can capture images of the driver's hand as the driver performs the motion; the device can then analyze the images to detect the clockwise motion, and in response detecting the motion, the device can cause the media track to fast-forward), and wherein the application operation instruction comprises rewinding the first multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand (column 3, lines 13 to 20 - while the track is playing, when the user's hand moves in the gesture area in a counter-clockwise circular motion, the device can cause the point of play to reverse backward in the time sequence of the current track).
Claims 3 and 26 are rejected under 35 U.S.C. 103 as being unpatentable over Schwesig in view of Yuan and Kim, further in view of Weber et al. (US 9,377,860 B1 hereinafter Weber) and Baheti et al. (US 2019/0212436 A1 hereinafter Baheti).
As to dependent Claim 3, Schwesig, Yuan, and Kim teach all the limitations of claim 2. Schwesig further teaches wherein the plurality of applications include a first video player ([0019] The gesture may be recognized by the gesture component to increase volume; the gesture component then expose via the application programming interfaces to an application to increase the volume in rendering of digital content, e.g., audio or multimedia content; [0034] media gestures 218 may be defined as a gesture library 202 to control operations involving creation and/or rendering of media, such as digital images, digital audio, video, multimedia, and other types of digital content rendered by the computing device), wherein performing the application operation instruction comprises performing the application operation instruction based on the mid-air gesture and a location at which the mid-air gesture occurs ([0031] detection of the gesture 210 by the 3D object detection system 120 includes positioning of fingers of the user's 124 hand 212 in three-dimensional space as well as detection of movement, which in this instance involves rotational movement to the left 214 or right 216; [0032] the gesture module 122 accepts data as an input from the 3D object detection system 120 that describes this positioning and movement; the level of volume is raised or lowered by rotating to the left 214 or right 216, respectively - thus, performing instructions based on the mid-air gesture and a location at which the mid-air gesture occur).
However, Schwesig, Yuan, and Kim do not expressly teach wherein when the mid-air gesture is an upward translational move of the hand or an upward wave of the hand, the application operation instruction comprises turning the play volume up when the location at which the mid-air gesture occurs is close to the electronic device or comprises turning the display brightness up when the location at which the mid-air gesture occurs is close to a left side of the electronic device, and wherein when the mid-air gesture is a downward translational move of the hand or a downward wave of the hand, the application operation instruction comprises turning the play volume down if the location at which the mid-air gesture occurs is close to the electronic device or comprises turning the display brightness down if the location at which the mid-air gesture occurs is close to the left side of the electronic device.
In the same field of endeavor, Weber teaches wherein when the mid-air gesture is an upward translational move of the hand or an upward wave of the hand, the application operation instruction comprises turning the play volume up when the location at which the mid-air gesture occurs is close to the electronic device or comprises turning the display brightness up when the location at which the mid-air gesture occurs is close to a left side of the electronic device (column 8, lines 19 to 22 - while the track is playing , when the user's hand moves in a down-to-up motion with the palm open and facing upward, the device volume can be increased; column 6, lines 28 to 32 - the gesture area can be a particular range of distances such 0-12 inches away from the mobile device (i.e., close to the device)), and wherein when the mid-air gesture is a downward translational move of the hand or a downward wave of the hand, the application operation instruction comprises turning the play volume down if the location at which the mid-air gesture occurs is close to the electronic device or comprises turning the display brightness down if the location at which the mid-air gesture occurs is close to the left side of the electronic device (column 8, lines 25 to 30 - while the track is playing, when the user's hand moves in an up-to-down motion with the palm open and facing upward, the device volume can be decreased; column 6, lines 28 to 32 - the gesture area can be a particular range of distances such 0-12 inches away from the mobile device (i.e., close to the device)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein when the mid-air gesture is an upward translational move of the hand or an upward wave of the hand, the application operation instruction comprises turning the play volume up when the location at which the mid-air gesture occurs is close to the electronic device or comprises turning the display brightness up when the location at which the mid-air gesture occurs is close to a left side of the electronic device, and wherein when the mid-air gesture is a downward translational move of the hand or a downward wave of the hand, the application operation instruction comprises turning the play volume down if the location at which the mid-air gesture occurs is close to the electronic device or comprises turning the display brightness down if the location at which the mid-air gesture occurs is close to the left side of the electronic device, as taught by Weber into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow to operate the device where touch input is not possible or permitted, or where diverting their visual focus to the device is dangerous (such as when driving) (Weber, column 1, lines 20 to 24).
However, Schwesig, Yuan, Kim, and Weber fail to expressly teach wherein changing volume when the location at which the mid-air gesture occurs is close to the right side of the electronic device.
In the same field of endeavor, Baheti teaches wherein adjusting volume when the location at which the mid-air gesture occurs is close to a right side of the electronic device ([0105] right earphone configured to perform a right side vertical slider function of adjusting the volume in response to detecting the vertical sliding gesture).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein adjusting volume when the location at which the mid-air gesture occurs is close to the right side of the electronic device, as taught by Baheti into Schwesig, Yuan, Kim, and Weber. Doing so would be desirable because it would incorporate alternative means performing functions, thereby improving functionality of the portable device (Baheti [0002]).
As to dependent Claim 26, Schwesig, Yuan, Kim, and Weber teach all the limitations of claim 25. Weber further teaches wherein the plurality of applications include a first video player (column 4, lines 1 to 5 - a media player operating on the computing device; column 2, lines 55 to 65 - “media player” refers to as a software application used by users to listen to media; “media” refers to a song, video), wherein performing the application operation instruction comprises performing the application operation instruction based on the mid-air gesture and a location at which the mid-air gesture occurs (column 7, lines 1 to 32 - media content being presented via the device; the computing device determines the clockwise motion of the driver's hand based at least in part on data received from one or more sensors of the computing device; the computing device monitor the gesture area (i.e., location), and detect when the driver's hand comes within the gesture area; when the device detects the user's hand, the device can capture images of the driver's hand as the driver performs the motion; the device can then analyze the images to detect the clockwise motion, and in response detecting the motion, the device can cause the media track to fast-forward (i.e., response based on gesture)), wherein when the mid-air gesture is an upward translational move of the hand or an upward wave of the hand, the application operation instruction comprises turning the play volume up when the location at which the mid-air gesture occurs is close to the electronic device or comprises turning the display brightness up when the location at which the mid-air gesture occurs is close to a left side of the electronic device (column 8, lines 19 to 22 - while the track is playing , when the user's hand moves in a down-to-up motion with the palm open and facing upward, the device volume can be increased; column 6, lines 28 to 32 - the gesture area can be a particular range of distances such 0-12 inches away from the mobile device (i.e., close to the device)), and wherein when the mid-air gesture is a downward translational move of the hand or a downward wave of the hand, the application operation instruction comprises turning the play volume down if the location at which the mid-air gesture occurs is close to the electronic device or comprises turning the display brightness down if the location at which the mid-air gesture occurs is close to the left side of the electronic device (column 8, lines 25 to 30 - while the track is playing, when the user's hand moves in an up-to-down motion with the palm open and facing upward, the device volume can be decreased; column 6, lines 28 to 32 - the gesture area can be a particular range of distances such 0-12 inches away from the mobile device (i.e., close to the device)).
However, Schwesig, Yuan, Kim, and Weber fail to expressly teach wherein changing volume when the location at which the mid-air gesture occurs is close to a right side of the electronic device.
In the same field of endeavor, Baheti teaches wherein adjusting volume when the location at which the mid-air gesture occurs is close to a right side of the electronic device ([0105] right earphone configured to perform a right side vertical slider function of adjusting the volume in response to detecting the vertical sliding gesture).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein adjusting volume when the location at which the mid-air gesture occurs is close to a right side of the electronic device, as taught by Baheti into Schwesig, Yuan, Kim, and Weber. Doing so would be desirable because it would incorporate alternative means performing functions, thereby improving functionality of the portable device (Baheti [0002]).
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Schwesig in view of Yuan and Kim, further in view of Lapidot et al. (US 2015/0074532 A1 hereinafter Lapidot).
As to dependent Claim 5, Schwesig, Yuan, and Kim teach all the limitations of claim 2. However, Schwesig, Yuan, and Kim do not expressly teach wherein the plurality of applications include a second video player and wherein the application operation instruction comprises switching the first multimedia file to a second multimedia file when the mid-air gesture is an upward translational move of the hand or an upward wave of the hand, and wherein the application operation instruction comprises switching the first multimedia file to a third multimedia file when the mid-air gesture is a downward translational move of the hand or a downward wave of the hand.
In the same field of endeavor, Lapidot teaches wherein the plurality of applications include a second video player and wherein the application operation instruction comprises switching the first multimedia file to a second multimedia file when the mid-air gesture is an upward translational move of the hand or an upward wave of the hand, and wherein the application operation instruction comprises switching the first multimedia file to a third multimedia file when the mid-air gesture is a downward translational move of the hand or a downward wave of the hand ([0005] The “swipe up” gesture causes a selected panel in a grid of panels displayed by the video management program to switch places a panel above. The “swipe down” gesture causes a selected panel in a grid of panels displayed by the video management program to switch places a panel below).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the plurality of applications include a second video player and wherein the application operation instruction comprises switching the first multimedia file to a second multimedia file when the mid-air gesture is an upward translational move of the hand or an upward wave of the hand, and wherein the application operation instruction comprises switching the first multimedia file to a third multimedia file when the mid-air gesture is a downward translational move of the hand or a downward wave of the hand, as taught by Lapidot into Schwesig, Yuan, and Kim. Doing so would be desirable because it would provide a faster, easier and more intuitive method for controlling video management software (Lapidot [0003]).
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Schwesig in view of Yuan and Kim, further in view of Weber et al. (US 9,377,860 B1 hereinafter Weber) and Harvey et al. (US 2018/0173323 A1 hereinafter Harvey).
As to dependent Claim 6, Schwesig, Yuan, and Kim teach all the limitations of claim 2. However, Schwesig, Yuan, and Kim do not expressly teach wherein the application operation instruction comprises switching the first multimedia file to a second multimedia file when the mid-air gesture is a rightward translational move of the hand or a rightward wave of the hand, and wherein the application operation instruction comprises switching the first multimedia file to a third multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand.
In the same field of endeavor, Weber teaches wherein the application operation instruction comprises switching the first multimedia file to a second multimedia file when the mid-air gesture is a rightward translational move of the hand or a rightward wave of the hand (column 3, lines 39 to 43 - the user can perform a gesture to skip to a next track; while the track is playing, when the user's hand moves in a left-to-right motion, the device can cause the point of play to move to the beginning of the next track within the media), and wherein the application operation instruction comprises switching the first multimedia file to a third multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand (column 3, lines 43 to 47 - when the user's hand moves in a right-to-left motion in the gesture area, the device can cause the point of play to move to the beginning of the previous track).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the application operation instruction comprises switching the first multimedia file to a second multimedia file when the mid-air gesture is a rightward translational move of the hand or a rightward wave of the hand, and wherein the application operation instruction comprises switching the first multimedia file to a third multimedia file when the mid-air gesture is a leftward translational move of the hand or a leftward wave of the hand, as taught by Weber into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow to operate the device where touch input is not possible or permitted, or where diverting their visual focus to the device is dangerous (such as when driving) (Weber, column 1, lines 20 to 24).
However, Schwesig, Yuan, Kim, and Weber fail to expressly teach wherein the plurality of applications include a gallery.
In the same field of endeavor, Harvey teaches wherein the plurality of applications include a gallery ([0119] Swipes to the left or right in within a gesture space could be used to (1) switch a window rendered within a virtual desktop, (2) enable/disable the display of a virtual keyboard, (3) based on the context, move forward/backward in a video, next/previous song, next/previous photo in the slideshow, etc.).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein the plurality of applications include a gallery, as taught by Harvey into Schwesig, Yuan, Kim, and Weber. Doing so would be desirable because it would provide improvement when operating within virtualized environments using physical interface device (Harvey [0004]).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Schwesig in view of Yuan and Kim, further in view of Smith (US 2016/0188181 A1).
As to dependent Claim 15, Schwesig, Yuan, and Kim teach all the limitations of claim 1. Schwesig further teaches mid-air gesture ([0029] the 3D object detection system 120 may detect an object and movement of the object. Data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; [0017] a gesture component incorporates techniques usable to detect gestures, such as a touchpad, touchscreen functionality, cameras; a three-dimensional object detection system may be employed in which gestures are detected in free space, without any attachment or peripheral device connected to a corresponding body part, such as a hand; the three-dimensional object detection system may leverage transmission and reception of radio signals to detect orientation and movement of an object (e.g., a body part of a user such as a hand) in three-dimensional space).
However, Schwesig, Yuan, and Kim do not expressly teach wherein upon determining, based on a preset first correspondence, that a gesture type corresponding to the mid-air gesture does not exist, determine that the mid-air gesture is an ineffective gesture, and determine an effective reference mid-air gesture based on a current application or a use habit of the user; and prompt the user with the effective reference mid-air gesture.
In the same field of endeavor, Smith teaches wherein upon determining, based on a preset first correspondence, that a gesture type corresponding to the mid-air gesture does not exist, determine that the mid-air gesture is an ineffective gesture ([0406] gesture predefined within an application or operating system; [0655] the default response to the inability to identify the gesture (i.e., ineffective gesture) may be to prompt the user to clarify their recent input), and determine an effective reference mid-air gesture based on a current application or a use habit of the user ([0655] the default response to the inability to identify the gesture may be to prompt the user to clarify their recent input; the user may be prompted by a pop-up interface which allows the user to select from identified potential gestures (i.e., effective reference gesture); [0731] probabilities may be assigned to potential gestures based upon historical gesture usage with similar context (i.e. application, selection mode, etc.)); and prompt the user with the effective reference mid-air gesture ([0655] the default response to the inability to identify the gesture (i.e., ineffective gesture) may be to prompt the user to clarify their recent input; the user may be prompted by a pop-up interface which allows the user to select from identified potential gestures (i.e., effective reference gesture), or indicate that none of the candidates are correct).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein upon determining, based on a preset first correspondence, that a gesture type corresponding to the mid-air gesture does not exist, determine that the mid-air gesture is an ineffective gesture, and determine an effective reference mid-air gesture based on a current application or a use habit of the user; and prompt the user with the effective reference mid-air gesture, as taught by Smith into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow for the gesture recognition process to be performed more efficiently, by considering the most likely gestures early in the process (Smith [0731]).
Claim 16 is rejected under 35 U.S.C. 103 as being unpatentable over Schwesig in view of Yuan and Kim, further in view of Smith (US 2016/0188181 A1) and Kjelsbak et al. (US 2013/0050118 A1 hereinafter Kjelsbak).
As to dependent Claim 16, Schwesig, Yuan, and Kim teach all the limitations of claim 1. Schwesig further teaches mid-air gesture ([0029] the 3D object detection system 120 may detect an object and movement of the object. Data that results from this detection is then used by the gesture module 122 to identify which of the plurality of gestures in the gesture library 202 correspond to the detected gesture; [0017] a gesture component incorporates techniques usable to detect gestures, such as a touchpad, touchscreen functionality, cameras; a three-dimensional object detection system may be employed in which gestures are detected in free space, without any attachment or peripheral device connected to a corresponding body part, such as a hand; the three-dimensional object detection system may leverage transmission and reception of radio signals to detect orientation and movement of an object (e.g., a body part of a user such as a hand) in three-dimensional space).
However, Schwesig, Yuan, and Kim do not expressly teach wherein upon determining, based on a preset first correspondence, that a gesture type corresponding to the mid-air gesture does not exist, determine that the mid-air gesture is an ineffective gesture, and determine an effective reference mid-air gesture based on a current application or a use habit of the user; determine a reference general-purpose input event corresponding to the effective reference mid-air gesture; determine a first operation based on the reference general-purpose input event and the current application; prompt the user whether to perform the first operation; and perform the first operation after an instruction indicating to perform the first operation from the user is detected.
In the same field of endeavor, Smith teaches wherein upon determining, based on a preset first correspondence, that a gesture type corresponding to the gesture does not exist, determine that the gesture is an ineffective gesture ([0406] gesture predefined within an application or operating system; [0655] the default response to the inability to identify the gesture (i.e., ineffective gesture) may be to prompt the user to clarify their recent input), and determine an effective reference gesture based on a current application or a use habit of the user ([0655] the default response to the inability to identify the gesture may be to prompt the user to clarify their recent input; the user may be prompted by a pop-up interface which allows the user to select from identified potential gestures (i.e., effective reference gesture); [0731] probabilities may be assigned to potential gestures based upon historical gesture usage with similar context (i.e. application, selection mode, etc.)); determine a reference general-purpose input event corresponding to the effective reference gesture ([0656] if the identity of the gesture being performed has been determined, it is then determined whether the gesture is meant to trigger some sort of transformation; [0659] the intermediate operations associated with the identified gesture are performed; [0662] each execution of intermediate gesture operations may trigger a display update/ rendering; [0666] once it has been determined that the gesture has ended, the final operations associated with the identified gesture are performed (i.e., determine input event)); determine a first operation based on the reference general-purpose input event and the current application ([0666] once it has been determined that the gesture has ended, the final operations associated with the identified gesture are performed; [0542] the association between gesture and operation may be context-dependent; the association depends on application context (e.g., which application is active, etc.)).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein upon determining, based on a preset first correspondence, that a gesture type corresponding to the gesture does not exist, determine that the gesture is an ineffective gesture, and determine an effective reference gesture based on a current application or a use habit of the user; determine a reference general-purpose input event corresponding to the effective reference gesture; determine a first operation based on the reference general-purpose input event and the current application, as taught by Smith into Schwesig, Yuan, and Kim. Doing so would be desirable because it would allow for the gesture recognition process to be performed more efficiently, by considering the most likely gestures early in the process (Smith [0731]).
However, Schwesig, Yuan, Kim, and Smith fail to expressly teach wherein prompt the user whether to perform the first operation; and perform the first operation after an instruction indicating to perform the first operation from the user is detected.
In the same field of endeavor, Kjelsbak teaches wherein prompt the user whether to perform the first operation ([0020] The user has previously set an “S” gesture as his “help me” gesture; the user may perform the “S” gesture on or in proximity of the screen of his mobile device. When the “S” gesture is detected and recognized by his mobile device, the user may be prompted with a page that overlays the mobile application and asks the user these questions: Would you like to “request for assistance,” “leave feedback,” “rate the application,” “report an error,” or “suggest improvement idea?”); and perform the first operation after an instruction indicating to perform the first operation from the user is detected ([0020] Responsive to selecting the “request for assistance” portion (e.g., button), the user may be called on his phone by a customer support agent who is informed regarding in which flow of the mobile application the user was at the time of the “help me” gesture being detected).
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention to have incorporated wherein prompt the user whether to perform the first operation and perform the first operation after an instruction indicating to perform the first operation from the user is detected, as taught by Kjelsbak into Schwesig, Yuan, Kim, and Smith. Doing so would be desirable because it would provide an efficient design of the user interface(s) of the application (Kjelsbak [0014]), thereby improving user experience.
Response to Arguments
Claim Objections: Applicant’s amendment have overcome the claim objections previously set forth.
35 U.S.C. §103: In the remarks, Applicant argues that: Weber fails to "display, on the display, an operation mark that relates to the mid-air gesture, wherein the operation mark simulates an actual touch operation performed by the user on a touchscreen to prompt the user with the general-purpose input event to which the electronic device responds, and wherein the operation mark is independent of the third-party application," as recited in amended independent claims 1, 9, and 20.
Applicant's arguments with respect to the 103 rejections have been considered, but are moot in view of new ground of rejection made under 35 U.S.C. § 103.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to REJI KARTHOLY whose telephone number is (571)272-3432. The examiner can normally be reached on Monday - Thursday 7:30 am - 3:30 pm.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Welch, can be reached at telephone number (571)272-7212. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center. Status information for published applications may be obtained from Patent Center. Status information for unpublished applications is available through Patent Center for authorized users only. Should you have questions about access to Patent Center, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) Form at https://www.uspto.gov/patents/uspto-automated- interview-request-air-form.
/REJI KARTHOLY/Primary Examiner, Art Unit 2143