Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
The instant application having Application No. 17435866 has a total of 21 claims pending in the application, of which claims 8-10, 12, and 14 have been cancelled or withdrawn.
Claim Rejections – 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-7, 11, 13, and 15-21 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
Step 1: Claim 1 is a machine type claims. Claims 11 is a process type claim. Claim 13 is a manufacture type claims. Therefore, claims 1-7, 11, 13, and 15-21 are directed to either a process, machine, manufacture or composition of matter.
As per claim 1,
2A Prong 1:
“Determine whether a change between the input data and preceding input data satisfies a predetermined threshold” A user mentally or with pencil and paper looks at the input and looks to see if its changed enough to be considered new input.
“when the change satisfies the predetermined threshold, generating, using a … output data for the input data… wherein the output data represents an estimation of a posture of a body part to which the tracking device is attached … to generate the output” A user mentally or with pencil and paper looks at the posture of their body and makes a determination of what that pose is.
combining the input data with a first state variable and providing combined input data” The user mentally or with pencil and paper performs the appropriate mathematics to combine the inputs and previous data.
“updating the first state variable with the second state variable” The user mentally or with pencil and paper updates the variable and notes the different postures of the body part by looking at them.
“when the change fails to satisfy the predetermined threshold, restraining an update of the first state variable associated with the neural network, and acquiring a latest output data…” The user mentally or with pencil and paper continues with the previous results when the input is not sufficient to generate new results.
“and processing either the output data generated when the change satisfies the predetermined threshold or the latest output data acquired when the change fails to satisfy the predetermined threshold to generate processed output data, wherein the processing comprises determining a plurality of postures associated with the body part” The user mentally or with pencil and paper updates the variable and notes the different postures of the body part by looking at them.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
“An information processing device”, “A storage unit”, “a processor”, “a tracking device” (mere instructions to apply the exception using a generic computer component);
“a neural network”, “the neural network comprises a long short-term memory (LSTM) block, and an output block comprising a plurality of layers”, “from the LSTM block”, “the plurality of layers of the output block” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: A generic off the shelf neural network with no additional detail or limitations that make it more than a generic neural network. Every neural network will have some form of layer, and include a means of outputting results);
“Acquiring input data corresponding to a series of pieces of sensing data… wherein each piece of sensing data of the series of pieces of sensing data is associated with a sequence number”, “inputting the combined input data… according to the sequence number associated with the input data wherein the first variable is associated with data previously processed… and indicates characteristics of a time series transition of the data previously processed… and wherein a second state variable is output … in response to the combined input data”, “Passing the second state variable through the … to generate the output data”, (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
“An information processing device”, “A storage unit”, “a processor”, “a tracking device” (mere instructions to apply the exception using a generic computer component)
“a neural network”, “the neural network comprises a long short-term memory (LSTM) block, and an output block comprising a plurality of layers”, “from the LSTM block”, “the plurality of layers of the output block” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: A generic off the shelf neural network with no additional detail or limitations that make it more than a generic neural network. Every neural network will have some form of layer, and include a means of outputting results);
“Acquiring input data corresponding to a series of pieces of sensing data… wherein each piece of sensing data of the series of pieces of sensing data is associated with a sequence number”, “inputting the combined input data… according to the sequence number associated with the input data wherein the first variable is associated with data previously processed… and indicates characteristics of a time series transition of the data previously processed… and wherein a second state variable is output … in response to the combined input data”, “Passing the second state variable through the … to generate the output data” (MPEP 2106.05(d)(II) indicate that merely “transmitting or receiving data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed acquiring step is well-understood, routine, conventional activity is supported under Berkheimer).
As per claims 2-7, 15-17, and 21 these claims contains additional mental steps of deciding to use the model, and additional generic machine learning aspects, and are rejected similarly to claim 1.
As per claim 18-19, these claims have additional mental steps similar to claim 1, and are rejected for similar reasons.
As per claim 20, this claim has additional mental steps, generic machine learning, and generic computer equipment similar to claim 1, and is rejected for similar reasons.
As per claim 11,
2A Prong 1:
“Determine whether a change between the input data and preceding input data satisfies a predetermined threshold” A user mentally or with pencil and paper looks at the input and looks to see if its changed enough to be considered new input.
“when the change satisfies the predetermined threshold, generating, using a … output data for the input data… wherein the output data represents an estimation of a posture of a body part to which the tracking device is attached … to generate the output” A user mentally or with pencil and paper looks at the posture of their body and makes a determination of what that pose is.
combining the input data with a first state variable and providing combined input data” The user mentally or with pencil and paper performs the appropriate mathematics to combine the inputs and previous data.
“updating the first state variable with the second state variable” The user mentally or with pencil and paper updates the variable and notes the different postures of the body part by looking at them.
“when the change fails to satisfy the predetermined threshold, restraining an update of the first state variable associated with the neural network, and acquiring a latest output data…” The user mentally or with pencil and paper continues with the previous results when the input is not sufficient to generate new results.
“and processing either the output data generated when the change satisfies the predetermined threshold or the latest output data acquired when the change fails to satisfy the predetermined threshold to generate processed output data, wherein the processing comprises determining a plurality of postures associated with the body part” The user mentally or with pencil and paper updates the variable and notes the different postures of the body part by looking at them.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
“a tracking device” (mere instructions to apply the exception using a generic computer component);
“a neural network”, “the neural network comprises an LSTM block, and an output block comprising a plurality of layers”, “from the LSTM block”, “the plurality of layers of the output block” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: A generic off the shelf neural network with no additional detail or limitations that make it more than a generic neural network. Every neural network will have some form of layer, and include a means of outputting results);
“Acquiring input data corresponding to a series of pieces of sensing data… wherein each piece of sensing data of the series of pieces of sensing data is associated with a sequence number”, “inputting the combined input data… according to the sequence number to which the input data is associated with, wherein the first variable is associated with data previously processed… and indicates characteristics of a time series transition of the data previously processed… and wherein the output data corresponds to a second state variable” (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
“a tracking device” (mere instructions to apply the exception using a generic computer component)
“a neural network”, “the neural network comprises an LSTM block, and an output block comprising a plurality of layers”, “from the LSTM block”, “the plurality of layers of the output block” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: A generic off the shelf neural network with no additional detail or limitations that make it more than a generic neural network. Every neural network will have some form of layer, and include a means of outputting results);
“Acquiring input data corresponding to a series of pieces of sensing data… wherein each piece of sensing data of the series of pieces of sensing data is associated with a sequence number”, “inputting the combined input data… according to the sequence number associated with the input data wherein the first variable is associated with data previously processed… and indicates characteristics of a time series transition of the data previously processed… and wherein a second state variable is output … in response to the combined input data”, “Passing the second state variable through the … to generate the output data” (MPEP 2106.05(d)(II) indicate that merely “transmitting and receiving data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed acquiring step is well-understood, routine, conventional activity is supported under Berkheimer).
As per claim 13,
2A Prong 1:
“Determine whether a change between the input data and preceding input data satisfies a predetermined threshold” A user mentally or with pencil and paper looks at the input and looks to see if its changed enough to be considered new input.
“when the change satisfies the predetermined threshold, generating, using a … output data for the input data… wherein the output data represents an estimation of a posture of a body part to which the tracking device is attached … to generate the output” A user mentally or with pencil and paper looks at the posture of their body and makes a determination of what that pose is.
combining the input data with a first state variable and providing combined input data” The user mentally or with pencil and paper performs the appropriate mathematics to combine the inputs and previous data.
“updating the first state variable with the second state variable” The user mentally or with pencil and paper updates the variable and notes the different postures of the body part by looking at them.
“when the change fails to satisfy the predetermined threshold, restraining an update of the first state variable associated with the neural network, and acquiring a latest output data…” The user mentally or with pencil and paper continues with the previous results when the input is not sufficient to generate new results.
“and processing either the output data generated when the change satisfies the predetermined threshold or the latest output data acquired when the change fails to satisfy the predetermined threshold to generate processed output data, wherein the processing comprises determining a plurality of postures associated with the body part” The user mentally or with pencil and paper updates the variable and notes the different postures of the body part by looking at them.
2A Prong 2: This judicial exception is not integrated into a practical application.
Additional elements:
“A non-transitory, computer readable storage medium”, “a tracking device” (mere instructions to apply the exception using a generic computer component);
“a neural network”, “the neural network comprises an LSTM block, and an output block comprising a plurality of layers”, “from the LSTM block”, “the plurality of layers of the output block” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: A generic off the shelf neural network with no additional detail or limitations that make it more than a generic neural network. Every neural network will have some form of layer, and include a means of outputting results);
“Acquiring input data corresponding to a series of pieces of sensing data… wherein each piece of sensing data of the series of pieces of sensing data is associated with a sequence number”, “inputting the combined input data… according to the sequence number associated with the input data wherein the first variable is associated with data previously processed… and indicates characteristics of a time series transition of the data previously processed… and wherein a second state variable is output … in response to the combined input data”, “Passing the second state variable through the … to generate the output data” (Adding insignificant extra-solution activity to the judicial exception - see MPEP 2106.05(g)).
2B: The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Additional elements:
“A non-transitory, computer readable storage medium”, “a tracking device” (mere instructions to apply the exception using a generic computer component);
“a neural network”, “the neural network comprises an LSTM block, and an output block comprising a plurality of layers”, “from the LSTM block”, “the plurality of layers of the output block” (Adding the words “apply it” (or an equivalent) with the judicial exception, or mere instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea - see MPEP 2106.05(f) – Examiner’s note: A generic off the shelf neural network with no additional detail or limitations that make it more than a generic neural network. Every neural network will have some form of layer, and include a means of outputting results);
“Acquiring input data corresponding to a series of pieces of sensing data… wherein each piece of sensing data of the series of pieces of sensing data is associated with a sequence number”, “inputting the combined input data… according to the sequence number associated with the input data wherein the first variable is associated with data previously processed… and indicates characteristics of a time series transition of the data previously processed… and wherein a second state variable is output … in response to the combined input data”, “Passing the second state variable through the … to generate the output data” (MPEP 2106.05(d)(II) indicate that merely “transmitting or receiving data” is a well‐understood, routine, conventional function when it is claimed in a merely generic manner (as it is in the present claim). Thereby, a conclusion that the claimed acquiring step is well-understood, routine, conventional activity is supported under Berkheimer).
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-7, 11, 13, and 15-21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
As per claims 1, 11, and 13, each of these claims call for two optional portions of the claim. The first, “when the change satisfies the predetermined threshold…” then later calls for “combining the input data with a first state variable from the LSTM block and providing combined input data.” The claim then goes on to include the alternative optional step: “when the change fails to satisfy the predetermined threshold, restraining an update of the first state variable associated with the neural network….” In the alternative optional step,” the first state variable” is never created, it is a part of the first optional step, and not the alternative optional step. This causes the claim to be confusing, as the alternative step is dealing with a variable that is never created when that step is initiated, and this causes the claim to be rejected under U.S.C. 112(b) for failing to particularly point out and claim the intended invention.
As per claims 1-7 and 15-21, these claims are rejected as being dependent on a claim rejected under U.S.C. 112(b).
As per claim 15, this claim calls for “the first intermediate layer”, “the second intermediate layer” and “the output layer.” There is insufficient antecedent basis for these limitations in the claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-4, 9, 11, and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Rasmussen et al (US 20140126759 A1) in view of Zhou et al (“Finger-worn Device Based Hand Gesture Recognition Using Long Shot-term Memory”) and Chen et al (“motion Feature Augmented Recurrent Neural Network for Skeleton-Based Dynamic Hand Gesture Recognition”).
As per claims 1, 11, and 13, Rasmussen discloses, “An information processing device, comprising” (pg.5, particularly paragraph 0046; EN: this denotes the hardware to run the system).
“A storage unit storing instructions which, when executed by the processor, cause the information processing device to perform operations comprising” (pg.5, particularly paragraph 0046; EN: this denotes the hardware to run the system).
“acquiring input data corresponding to a series of pieces of sensing data” (Pg.3-4, particularly paragraph 0035; EN: this denotes taking data in from sensors over time sequences). “measured by a tracking device” (Pg.3-4, particularly paragraph 0035; EN: this denotes the use of sensor electrodes).
“Determining whether a change between the input data and preceding input data satisfies a predetermined threshold” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands. Here small changes from the previous gesture I.e. less than the minimum distance, will not trigger new input).
“When the change satisfies the predetermined threshold, generating, using a neural network” (pg.4, particularly paragraph 0036; EN: this denotes recognizing via a neural network). “Output data from the input data, wherein the output data represents an estimation of a posture of a body part to which the tracking device is attached” (Pg.4, particularly paragraph 0036; EN: this denotes using the data to determine gestures (i.e. postures) of the attached body part). “wherein the neural network comprises a … block and an output block …” (Pg.4, particularly paragraph 0036; EN: Any neural network will inherently have multiple layers and an output layer (i.e. output block) of some kind). “the generating the output comprising:” (Pg.4, particularly paragraph 0036; EN: this denotes using the data to determine gestures (i.e. postures) of the attached body part).
“inputting the … input data into the … block according to the sequence … associated with the input data …”(Pg.3-4, particularly paragraph 0035; EN: this denotes taking data in from sensors over time sequences).
“when the change fails to satisfy the predetermined threshold, restraining an update of the first state variable associated with the neural network” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands. Here small changes from the previous gesture I.e. less than the minimum distance, will not trigger new input). “and acquiring a latest output data generated by the neural network” (pg.4, particularly paragraph 0037; EN: this denotes continuous aspects of the system such as on/off, switching audio inputs, answering a phone call, etc which will be continued when no new command is issued).
“Processing either the output data generated when the change satisfies the predetermined threshold or the latest output data acquired when the change fails to satisfy the predetermined threshold to generate processed output data” ” (pg.4, particularly paragraph 0037; EN: this denotes continuous aspects of the system such as on/off, switching audio inputs, answering a phone call, etc which will be continued when no new command is issued). “wherein the processing comprises determining a plurality of postures associated with the body part” (pg.4, particularly paragraph 0036; EN: this denotes the system being able to recognize multiple gestures).
However, Rasmussen fails to explicitly disclose, “wherein each piece of sensing data of the series of pieces of sensing data is associated with a sequence number”, “wherein the neural network comprises a long short-term memory (LSTM) block”, “an output block comprising a plurality of layers”, “combining the input data with a first state variable from the LSTM block and providing combined input data”, “the combined input”, “sequence number”, “wherein the first state variable is associated with data previously processed by the neural network and indicates characteristics of a time series transition of the data previously processed by the neural network and wherein a second state variable is output from the LSTM block in response to the combined input data”, “passing the second state variable through the plurality of layers of the output block to generate the output data”, “updating the first state variable with the second state variable”
Zhou discloses, “wherein each piece of sensing data of the series of pieces of sensing data I s associated with a sequence number” and “Sequence number”(Pg.2069, section C, particularly equation 3 and associated paragraphs; EN: this denotes keeping track of each piece of a sequence by number).
“wherein the neural network comprises a long short-term memory (LSTM) block” (Pg.2070, particularly C1, Second paragraph until the end, C2, particularly the first two paragraphs; EN: this denotes the use of the LSTM recurrent neural network block).
“combining the input data with a first state variable from the LSTM block and providing combined input data”, “the combined input” (Pg.2070, particularly C1, Second paragraph until the end, C2, particularly the first two paragraphs; EN: this denotes the use of the LSTM recurrent neural network block, with the input at time ‘t’ being current input, and the t-1 inputs being previous inputs that are being used with the current inputs).
“wherein the first state variable is associated with data previously processed by the neural network and indicates characteristics of a time series transition of the data previously processed by the neural network and wherein a second state variable is output from the LSTM block in response to the combined input data”, “passing the second state variable through … output block to generate the output data”, “updating the first state variable with the second state variable” (Pg.2070, particularly C1, Second paragraph until the end, C2, particularly the first two paragraphs; EN: this denotes the use of the LSTM recurrent neural network block, with the input at time ‘t’ being current input, and the t-1 inputs being previous inputs that are being used with the current inputs. This operates continuously as the system operates, with new data being saved to replace old data and used later as determined by the system).
Chen discloses, “an output block comprising a plurality of layers” and “through the plurality of layers of the output block…” (Pg.2881, C2, figure 1 and associated paragraphs; EN: This denotes the output after the LSTM layer being a series of Fully connected layers).
Rasmussen and Zhou are analogous art because both involve gesture recognition.
Before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Zhou in order to make use of recurrent neural networks for gesture recognition.
The motivation for doing so would be to “effectively model the long-term temporal dependency in a sequence for classification… it has shown impressive performance and stability in sequence prediction problems like … wearable activity recognition” (Zhou, Pg.2068, C2, second paragraph) or in the case of Rasmussen, allow the system to use a Recurrent neural network in order to improve their gesture recognition.
Therefore before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Zhou in order to make use of recurrent neural networks for gesture recognition.
Chen and Rasmussen modified by Zhao are analogous art because both involve gesture recognition.
Before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Chen and Rasmussen modified by Zhao in order to make use of multiple layers in the output portion of the network.
The motivation for doing so would be to make use of “three FC layers and a softmax layer for class prediction” (Chen, Pg.2883, C1, section 4).
Therefore before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Chen and Rasmussen modified by Zhao in order to make use of multiple layers in the output portion of the network.
As per claim 2, Rasmussen discloses, “after acquiring the input data, determining whether to provide the input data to the neural network” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands).
As per claim 3, Rasmussen discloses, “wherein determining whether to provide the input data to the neural network comprises determining that the input data is to be provided to the neural network” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands).
Zhou discloses, “storing the output data” (Pg.2070, particularly C1, Second paragraph until the end, C2, particularly the first two paragraphs; EN: this denotes the use of the LSTM recurrent neural network block, with the input at time ‘t’ being current input, and the t-1 inputs being previous inputs that are being used with the current inputs. This operates continuously as the system operates, with new data being saved to replace old data and used later as determined by the system).
As per claim 4, Rasmussen discloses, “wherein determining whether to provide the input data to the neural network comprises determining that the input data is not to be provided to the neural network and the operations further comprising” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands).
“retrieving stored output data and using the stored output data to execute processing on the stored output data” (Pg.4, particularly paragraph 0037; EN: this denotes the system issuing commands and controlling the device with the gestures. The previous gesture will continue to be executed despite ignored movements, such as changing gain, changing volume, changing program selection, etc).
As per claim 17, Rasmussen discloses, “wherein determining whether the change satisfies the predetermined threshold comprises identifying an absolute value of a difference between an output of the neural network corresponding to the input data and a neural network output immediately preceding said output” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands. Here small changes from the previous gesture I.e. less than the minimum distance, will not trigger new input. This is absolute value as it is a distance measurement from the previous point, which will never be negative).
As per claim 19, Rasmussen discloses, “…when the change satisfies the predetermined threshold…” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands. Here small changes from the previous gesture I.e. less than the minimum distance, will not trigger new input).
“wherein acquiring the latest output data when the change fails to satisfy the predetermined threshold comprises retrieving the most recently generated output data from the storage unit” (pg.4, particularly paragraph 0037; EN: this denotes continuous aspects of the system such as on/off, switching audio inputs, answering a phone call, etc which will be continued when no new command is issued).
Zhou discloses, “storing the output data with the output data generated … as a most recently generated output data in the storage unit” (Pg.2070, particularly C1, Second paragraph until the end, C2, particularly the first two paragraphs; EN: this denotes the use of the LSTM recurrent neural network block, with the input at time ‘t’ being current input, and the t-1 inputs being previous inputs that are being used with the current inputs. This operates continuously as the system operates, with new data being saved to replace old data and used later as determined by the system).
As per claim 20, Zhou discloses, “wherein the tracking device is a tracker attached to an end of a body comprising the body part, and wherein the output data generated by the neural network indicates an estimation result of a posture of a body part situated closer to a center of the body than the end of the body” (pg.2067, particularly C1, last paragraph, C2, first paragraph; EN: this denotes finger sensors that sense not only the finger itself, but the hand, which is closer to the body than the finger).
As per claim 21, Chen discloses, “Wherein the plurality of layers of the output block comprise a first intermediate layer, a second intermediate layer, and an output layer a plurality of layers, and generating the output data further comprises” (Pg.2881, C2, figure 1 and associated paragraphs; EN: This denotes the output after the LSTM layer being a series of Fully connected layers).
“inputting into the first intermediate layer of the output block; (Pg.2881, C2, figure 1 and associated paragraphs; EN: This denotes the output after the LSTM layer being a series of Fully connected layers each one connecting into the other until it reaches the output).
“inputting an output of the first intermediate layer into the second intermediate layer of the output block” (Pg.2881, C2, figure 1 and associated paragraphs; EN: This denotes the output after the LSTM layer being a series of Fully connected layers each one connecting into the other until it reaches the output).
“inputting an output of the second intermediate layer into the output layer” (Pg.2881, C2, figure 1 and associated paragraphs; EN: This denotes the output after the LSTM layer being a series of Fully connected layers each one connecting into the other until it reaches the output).
Claim Rejections - 35 USC § 103
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Rasmussen et al (US 20140126759 A1) in view of Zhou et al (“Finger-worn Device Based Hand Gesture Recognition Using Long Shot-term Memory”) and further in view of Zhu et al (“Online Hand Gesture Recognition using Neural Network Based Segmentation”).
As per claim 5, Rasmussen discloses, “wherein determining whether to provide the input data to the neural network comprises” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands).
However, Rasmussen fails to explicitly disclose, “determining, whether to restrain an update of a state associated with the neural network based on an output generated by a machine learning model when the input data is provided to the machine learning model.”
Zhu discloses, “determining, whether to restrain an update of a state associated with the neural network based on an output generated by a machine learning model when the input data is provided to the machine learning model” (Pg.2416, C2, Section A; Pg.2417, C1, before section B; EN: this denotes the use of a neural network to detect intended gestures).
Rasmussen and Zhu are analogous art because both involve gesture recognition.
Before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Zhu in order to allow a machine learning algorithm to be used to determine whether a gesture has been received.
The motivation for doing so would be to “spot gestures from daily non-gesture movements” (Zhu, Pg.2416, C2, section A, first paragraph) or in the case of Rasmussen, allow the system to use a machine learning algorithm to determine when a gesture is intended for the system and when it is not in order to avoid false positives.
Therefore before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Zhu in order to allow a machine learning algorithm to be used to determine whether a gesture has been received.
As per claim 16, Rasmussen discloses, “controlling whether to restrain the update of the first state variable based on a determination result output…” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands).
However, Rasmussen fails to explicitly disclose, “inputting the input data into an input determination model that is a trained machine learning model distinct from the neural network”, and “… from the input determination model in response to the input data.”
Zhu discloses, “inputting the input data into an input determination model that is a trained machine learning model distinct from the neural network”, and “… from the input determination model in response to the input data” (Pg.2416, C2, Section A; Pg.2417, C1, before section B; EN: this denotes the use of a neural network to detect intended gestures).
Rasmussen and Zhu are analogous art because both involve gesture recognition.
Before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Zhu in order to allow a machine learning algorithm to be used to determine whether a gesture has been received.
The motivation for doing so would be to “spot gestures from daily non-gesture movements” (Zhu, Pg.2416, C2, section A, first paragraph) or in the case of Rasmussen, allow the system to use a machine learning algorithm to determine when a gesture is intended for the system and when it is not in order to avoid false positives.
Therefore before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Zhu in order to allow a machine learning algorithm to be used to determine whether a gesture has been received.
Claim Rejections - 35 USC § 103
Claims 6-8 are rejected under 35 U.S.C. 103 as being unpatentable over Rasmussen et al (US 20140126759 A1) in view of Zhou et al (“Finger-worn Device Based Hand Gesture Recognition Using Long Shot-term Memory”) and further in view of Alameh et al (US 20100295781 A1).
As per claim 6, Rasmussen discloses, “wherein determining whether to provide the input data to the neural network comprises” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands).
However, Rasmussen fails to explicitly disclose, “Determining whether to restrain an update of a state associated with the neural network based on a change between the input data and a portion or all of previously acquired input data”
Alameh discloses, “Determining whether to restrain an update of a state associated with the neural network based on a change between the input data and a portion or all of previously acquired input data” (pg.19, particularly paragraphs 0149-0150; EN: this denotes considering the differences between the current input and the previous input when detecting a new gesture).
Rasmussen and Alameh are analogous art because both involve gesture recognition.
Before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Alameh in order to consider the immediately previous gesture when looking for new gestures.
The motivation for doing so would be to allow the system to “After an identification of a first gesture, the magnitude (absolute value) of a recognition threshold, a detection threshold and/or a clearance threshold… application to detection of a second gesture can be changed from a corresponding threshold applicable to the first gesture” (Alameh, Pg.19, paragraph 0150) or in the case of Rasmussen, allow the system to consider previous gestures when attempting to detect new gestures for the system.
Therefore before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Alameh in order to consider the immediately previous gesture when looking for new gestures.
As per claim 7, Rasmussen discloses, “wherein determining whether to provide the input data to the neural network comprises” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands).
However, Rasmussen fails to explicitly disclose, “determining whether to restrain an update of a state associated with the neural network based on a change between element sin the input data and element sin previously acquired input data.”
Alameh discloses, “determining whether to restrain an update of a state associated with the neural network based on a change between element sin the input data and element sin previously acquired input data” (pg.19, particularly paragraphs 0149-0150; EN: this denotes considering the differences between the current input and the previous input when detecting a new gesture).
Rasmussen and Alameh are analogous art because both involve gesture recognition.
Before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Alameh in order to consider the immediately previous gesture when looking for new gestures.
The motivation for doing so would be to allow the system to “After an identification of a first gesture, the magnitude (absolute value) of a recognition threshold, a detection threshold and/or a clearance threshold… application to detection of a second gesture can be changed from a corresponding threshold applicable to the first gesture” (Alameh, Pg.19, paragraph 0150) or in the case of Rasmussen, allow the system to consider previous gestures when attempting to detect new gestures for the system.
Therefore before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Alameh in order to consider the immediately previous gesture when looking for new gestures.
As per claim 18, Rasmussen discloses, “wherein determining whether the change satisfies the predetermined threshold is evaluated based on a change from the preceding input data …” (pg.4, particularly paragraph 0040; EN: this denotes ignoring detected movement that is outside the minimum and maximum range of detected gestures in order to avoid triggering of control commands).
However, Rasmussen fails to explicitly disclose, “… regarding a relative relation between a plurality of structural elements within the input data.”
Alameh discloses, “… regarding a relative relation between a plurality of structural elements within the input data” (pg.19, particularly paragraphs 0149-0150; EN: this denotes considering the differences between the current input and the previous input when detecting a new gesture).
Rasmussen and Alameh are analogous art because both involve gesture recognition.
Before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Alameh in order to consider the immediately previous gesture when looking for new gestures.
The motivation for doing so would be to allow the system to “After an identification of a first gesture, the magnitude (absolute value) of a recognition threshold, a detection threshold and/or a clearance threshold… application to detection of a second gesture can be changed from a corresponding threshold applicable to the first gesture” (Alameh, Pg.19, paragraph 0150) or in the case of Rasmussen, allow the system to consider previous gestures when attempting to detect new gestures for the system.
Therefore before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Alameh in order to consider the immediately previous gesture when looking for new gestures.
Claim Rejections - 35 USC § 103
Claims 15 is rejected under 35 U.S.C. 103 as being unpatentable over Rasmussen et al (US 20140126759 A1) in view of Zhou et al (“Finger-worn Device Based Hand Gesture Recognition Using Long Shot-term Memory”) and further in view of Zhao et al (“Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors”).
As per claim 15, Rasmussen fails to explicitly disclose, “wherein the first intermediate layer and the second intermediate layer are fully connected layers having a rectified linear function (RELU) as an activation function, and the output layer has a linear function as an activation function”
However, Zhao discloses, “wherein the first intermediate layer and the second intermediate layer are fully connected layers having a rectified linear function (RELU) as an activation function” (pg.4, particularly C1, first paragraph; EN: this denotes using fully connected layers with RELU). ,” and the output layer has a linear function as an activation function” (Pg.2, particularly C2, section 2; EN: this denotes the output layer being linear).
Rasmussen and Zhao are analogous art because both involve gesture recognition.
Before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Zhao in order to make use of RELU with fully connected layers.
The motivation for doing so would be to allow the system to “reduce the number of features in half” (Zhao, Pg.4, C1, first paragraph) or in the case of Rasmussen, allow the system to be simplified and focus on less features in order to produce the intended results.
Therefore before the effective filing date it would have been obvious to one skilled in the art of gesture recognition to combine the work of Rasmussen and Zhao in order to make use of RELU with fully connected layers.
Response to Arguments
Applicant's arguments with respect to claims 1-7, 11, 13, and 15-21 have been considered but are moot in view of the new ground(s) of rejection.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BEN M RIFKIN whose telephone number is (571)272-9768. The examiner can normally be reached Monday-Friday 9 am - 5 pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Alexey Shmatov can be reached at (571) 270-3428. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/BEN M RIFKIN/Primary Examiner, Art Unit 2123
Prosecution Insights