Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This office action is in response to the preliminary amendment filed 3/31/22. As directed by the amendment: claims 1-12 have been amended, claims 13-20 have been added, and no claims have been cancelled. As such, claims 1-20 are pending in the instant application.
Information Disclosure Statement
The information disclosure statement filed 3/31/22 fails to comply with 37 CFR 1.98(a)(3)(i) because it does not include a concise explanation of the relevance, as it is presently understood by the individual designated in 37 CFR 1.56(c) most knowledgeable about the content of the information, of each reference listed that is not in the English language. It has been placed in the application file, but the information referred to therein has not been considered.
Note: the crossed-through reference is not being considered at this time as it has not been provided with an English language translation.
Claim Objections
Claims 1 and 3 are objected to because of the following informalities:
Regarding claim 1, the language “a patient” (line 3) is objected to as ‘a patient’ has already been set forth in line 1; Examiner suggests amending to read –the patient--.
Regarding claim 3, the language “a degree for synchronization” (line 1-2) is objected to for a typographical/grammatical error; it appears this language should read –a degree of synchronization--.
Appropriate correction is required.
Claim Interpretation
The language “Computer-readable non-transient medium” (claim 12) is being interpreted as being equivalent to the language/phrase ‘non-transitory’ and thus not including non-statutory, transitory forms of signal transmission.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, the language “the video frame” (line 6 and 8) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (line 5) this language is referring to.
Regarding claim 1, the language “a so-called pressing probability” (line 5-6, emphasis added) is unclear as the metes and bounds of this limitation cannot be ascertain, in particular what limitation is intended by the term ‘so-called’.
Regarding claim 1, the language “a so-called compression probability” (line 7-8, emphasis added) is unclear as the metes and bounds of this limitation cannot be ascertain, in particular what limitation is intended by the term ‘so-called’.
Regarding claim 1, the language “a so-called inhalation probability” (line 10, emphasis added) is unclear as the metes and bounds of this limitation cannot be ascertain, in particular what limitation is intended by the term ‘so-called’.
Regarding claim 1, the language “the audio segment” (line 11) is unclear as it is not known which, one or a particular subset, of the plurality of audio segments (line 10) this language is referring to.
Claim 1 recites the limitation "the aerosol stream" in line 12. There is insufficient antecedent basis for this limitation in the claim.
Claim 1 recites the limitation "the actuation" in line 13. There is insufficient antecedent basis for this limitation in the claim.
Regarding claim 5, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Claim 5 recites the limitation "the amplitude" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Regarding claim 6, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Regarding claim 7, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Regarding claim 8, the language “the audio segment” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of audio segments (claim 1 line 10) this language is referring to.
Claim 9 recites the limitation "the steps consisting of calculating the pressing, compression and inhalation probabilities on later audio segments and video frames" in line 2-3. There is insufficient antecedent basis for this limitation in the claim.
Regarding claim 11, the language “the video frame” (line 6 and 8) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (line 5) this language is referring to.
Regarding claim 11, the language “a so-called pressing probability” (line 5-6, emphasis added) is unclear as the metes and bounds of this limitation cannot be ascertain, in particular what limitation is intended by the term ‘so-called’.
Regarding claim 11, the language “a so-called compression probability” (line 7-8, emphasis added) is unclear as the metes and bounds of this limitation cannot be ascertain, in particular what limitation is intended by the term ‘so-called’.
Regarding claim 11, the language “a so-called inhalation probability” (line 10, emphasis added) is unclear as the metes and bounds of this limitation cannot be ascertain, in particular what limitation is intended by the term ‘so-called’.
Regarding claim 11, the language “the audio segment” (line 11) is unclear as it is not known which, one or a particular subset, of the plurality of audio segments (line 10) this language is referring to.
Claim 11 recites the limitation "the aerosol stream" in line 11. There is insufficient antecedent basis for this limitation in the claim.
Claim 11 recites the limitation "the actuation" in line 12. There is insufficient antecedent basis for this limitation in the claim.
Regarding claim 13, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Claim 13 recites the limitation "the amplitude" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Regarding claim 14, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Claim 14 recites the limitation "the amplitude" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Regarding claim 15, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Claim 15 recites the limitation "the amplitude" in line 6. There is insufficient antecedent basis for this limitation in the claim.
Regarding claim 16, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Regarding claim 17, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Regarding claim 18, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Regarding claim 19, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Regarding claim 20, the language “the video frame” (line 3) is unclear as it is not known which, one or a particular subset, of the plurality of video frames (claim 1 line 5) this language is referring to.
Claims 2-4, 10, and 12 are rejected based on dependency on a rejected claim.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to a judicial exception (i.e., law of nature, natural phenomenon, or an abstract idea) without significantly more.
Each of claims 1-20 have been analyzed to determine whether it is directed to any judicial exceptions
Step 2A, Prong 1
Each of claims 1-20 recite at least one step or instruction for observing the use of a pressurized metered-dose inhaler (MDI) and determination of pressing/compression, inhalation, and degree of synchronization between actuation and inhalation, which is grouped as a mental process under the 2019 PEG. Accordingly, each of claims 5-12 recites an abstract idea.
Specifically, claims 1 and 11 recite obtaining video and audio signals of use of the MDI (observation, which is a mental process under the 2019 PEG); calculating pressing/compression and inhalation probabilities, determining synchronization between inhalation actuation (judgement or evaluation, which is a mental process under the 2019 PEG); and signaling proper/improper use of the MDI (judgement or evaluation, which is a mental process under the 2019 PEG).
Accordingly, as indicated above, the above identified claim recites an abstract idea.
Further, dependent claims 2-10 and 12-20 merely include limitations that either further define the abstract idea (and thus don’t make the abstract idea any less abstract) or amount to no more than generally linking the use of the abstract idea to a particular technological environment or field of use because they’re merely incidental or token additions to the claims (obtaining video signal/frames and audio signal/segments being extra solution activity).
Accordingly, each of claims 13-19 recites an abstract idea.
Step 2A, Prong 2
The above-identified abstract idea in each independent claims 1 and 11 (and their respective dependent claims) is not integrated into a practical application under the 2019 PEG because they do not include any additional elements or steps which, either alone or in combination, generally link the use of the above-identified abstract idea to a particular technological environment or field of use. In light of the disclosure it is understood that a processor(s) are performing the steps of obtaining video and audio signal, calculating pressing/compression and inhalation probabilities and determining degree of synchronization between inhalation and actuation, such a processor(s) is a generic computer element which do not improve the functioning of a computer, or any other technology or technical field. Nor would the controller serve to apply the above-identified abstract idea with, or by use of, a particular machine, effect a transformation or apply or use the above-identified abstract idea in some other meaningful way beyond generally linking the use thereof to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception. Furthermore, the above-identified processor(s) does not add a meaningful limitation to the abstract idea because it amounts to simply implementing the abstract idea on a computer. Additionally, Applicant’s specification does not include any discussion of how the claimed invention provides a technical improvement realized by these claims over the prior art or any explanation of a technical problem having an unconventional technical solution that is expressed in these claims. That is, like Affinity Labs of Tex. v. DirecTV, LLC, the specification fails to provide sufficient details regarding the manner in which the claimed invention accomplishes any technical improvement or solution.
For at least these reasons, the abstract idea identified above in independent claims 1 and 11 (and their respective dependent claims) is not integrated into a practical application under 2019 PEG.
Step 2B
None of claims 1-20 include additional elements that are sufficient to amount to significantly more than the abstract idea for at least the following reasons.
These claims do not appear to include additional elements that amount to significantly more than the judicial exception. As discussed above, Applicant’s specification/claim 11 sets forth that a processor(s) performs the claimed method; however, such a processor(s) is reasonably construed as a generic computer device. Like SAP America vs Investpic, LLC (Federal Circuit 2018), it is clear, from the claims themselves and the specification, that these limitations require no improved computer resources, just already available computers, with their already available basic functions, to use as tools in executing the claimed process. Furthermore, Applicant’s specification does not describe any special programming or algorithms required for the processor(s). This lack of disclosure is acceptable under 35 U.S.C. §112(a) since this hardware performs non-specialized functions known by those of ordinary skill in the computer arts. By omitting any specialized programming or algorithms, Applicant's specification essentially admits that this hardware is conventional and performs well understood, routine and conventional activities in the computer industry or arts. In other words, Applicant’s specification demonstrates the well-understood, routine, conventional nature of the above-identified additional elements because it describes these additional elements in a manner that indicates that the additional elements are sufficiently well-known that the specification does not need to describe the particulars of such additional elements to satisfy 35 U.S.C. § 112(a) (see Berkheimer memo from April 19, 2018, (III)(A)(1) on page 3). Adding hardware that performs “‘well understood, routine, conventional activit[ies]’ previously known to the industry” will not make claims patent-eligible (TLI Communications).
Such a recitation of a processor(s) in claim 11-12 amounts to mere instructions to implement the abstract idea on a computer. Simply using a computer or other machinery in its ordinary capacity for economic or other tasks (e.g. to obtain, manipulate or display data) or simply adding a general purpose computer or computer components after the fact to an abstract idea (e.g., a fundamental economic practice or mathematical equation) does not provide significantly more. See Affinity Labs v. DirecTV, 838 F.3d 1253, 1262, 120 USPQ2d 1201, 1207 (Fed. Cir. 2016) (cellular telephone); and TLI Communications LLC v. AV Auto, LLC, 823 F.3d 607, 613, 118 USPQ2d 1744, 1748 (Fed. Cir. 2016) (computer server and telephone unit). Moreover, implementing an abstract idea on a generic computer, does not add significantly more, similar to how the recitation of the computer in the claim in Alice amounted to mere instructions to apply the abstract idea of intermediated settlement on a generic computer.
A claim that purports to improve computer capabilities or to improve an existing technology may provide significantly more. McRO, Inc. v. Bandai Namco Games Am. Inc., 837 F.3d 1299, 1314-15, 120 USPQ2d 1091, 1101-02 (Fed. Cir. 2016); and Enfish, LLC v. Microsoft Corp., 822 F.3d 1327, 1335-36, 118 USPQ2d 1684, 1688-89 (Fed. Cir. 2016). However, a technical explanation as to how to implement the invention should be present in the specification for any assertion that the invention improves upon conventional functioning of a computer, or upon conventional technology or technological processes. That is, the disclosure must provide sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. Here, Applicant’s specification does not include any discussion of how the claimed invention provides a technical improvement realized by these claims over the prior art or any explanation of a technical problem having an unconventional technical solution that is expressed in these claims. Instead, as in Affinity Labs of Tex. v. DirecTV, LLC 838 F.3d 1253, 1263-64, 120 USPQ2d 1201, 1207-08 (Fed. Cir. 2016), the specification fails to provide sufficient details regarding the manner in which the claimed invention accomplishes any technical improvement or solution.
For at least the above reasons, claims 1-20 are directed to applying an abstract idea (e.g., mental process or certain method of organizing human activity) on a general purpose computer without (i) improving the performance of the computer itself (as in McRO, Bascom and Enfish), or (ii) providing a technical solution to a problem in a technical field (as in DDR). In other words, none of claims 1-20 provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that these claims amount to significantly more than the abstract idea itself.
Taking the additional elements individually and in combination, the additional elements do not provide significantly more. Specifically, when viewed individually, the above-identified additional elements in independent claims 1 and 11 (and their dependent claims) do not add significantly more because they are simply an attempt to limit the abstract idea to a particular technological environment. That is, neither the general computer elements nor any other additional element adds meaningful limitations to the abstract idea because these additional elements represent insignificant extra-solution activity. When viewed as a combination, these above-identified additional elements simply instruct the practitioner to implement the claimed functions with well-understood, routine and conventional activity specified at a high level of generality in a particular technological environment. As such, there is no inventive concept sufficient to transform the claimed subject matter into a patent-eligible application. As such, the above-identified additional elements, when viewed as whole, do not provide meaningful limitations to transform the abstract idea into a patent eligible application of the abstract idea such that the claims amount to significantly more than the abstract idea itself. Thus, claims 1-20 merely apply an abstract idea to a computer and do not (i) improve the performance of the computer itself (as in Bascom and Enfish), or (ii) provide a technical solution to a problem in a technical field (as in DDR). Therefore, none of the claims 1-20 amounts to significantly more than the abstract idea itself. Accordingly, claims 1-20 are not patent eligible and rejected under 35 U.S.C. 101 as being directed to abstract ideas implemented on a generic computer in view of the Supreme Court Decision in Alice Corporation Pty. Ltd. v. CLS Bank International, et al. and 2019 PEG.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 5-7, 9, 11, 19, and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hanina et al. (2013/0063579) in view of Bonney et al. (6,752,145) and Huang et al. (2020/0381111).
Regarding claim 1, Hanina discloses a computer-implemented method for tracking use, by a patient, of a pressurized MDI (see abstract, para. 0002, 0018-0024, 0038 for example) including the steps: obtaining a video signal and an audio signal of the patient using the MDI (see para. 0007, 0014, 0046, 0066 regarding MDI, para. 0022, 0038, 0041 regarding obtaining video and audio signals of a user using the inhaler); calculating from the video signal at least one of a so-called pressing probability that an actuating finger of the patient is actuating the MDI and a so-called compression probability that the MDI is in a actuated (see para. 0023 which discloses tracking/determining of movements associated with actuation of the inhaler including finger actuation which is indicative of MDI actuation, see also para. 0041, 0048, 0056-0057, 0059; see para. 0056, 0070, and 0080 which discloses determining levels of confidence of proper actuation which is a probability of actuation), calculating from the audio signal a so-called inhalation probability of the patient performing, in the audio signal, an inspiration combined with the aerosol stream (see para. 0023 which discloses calculating patient use/inhalation of the inhaler from audio signal, see also 0056 and 0066; see para. 0056, 0070, and 0080 which discloses determining levels of confidence of proper use including probability of inhalation), determining synchronization between actuation of the inhaler and an inspiration by the patient from the pressing/compression and halation probabilities to the same instants in time (see para. 0042, 0022-0023, 0041, see also abstract and para. 0002 which discloses analyzing to determine proper administration which includes synchronization of actuation and inhalation), and accordingly issuing to the patient a signal of proper use or misuse of the inhaler (see para. 0024, 0045). Hanina is silent as to the actuation of the MDI being finger pressing on a trigger member and/or compressed state of the MDI and to determine a degree of synchronization between inhalation and actuation; however, Bonney teaches a similar method which includes these limitations (see Bonney col. 3 ln. 16-25 regarding determining degree of synchronization between actuation and inhalation, col. 6 ln. 35-41 regarding finger pressing/compression of the MDI for actuation of an MDI). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Hanina method to explicitly determine a degree of synchronization between inhalation and actuation and for the MDI to be of the finger pressing/compression state type MDI, as taught by Bonney, as this is a well-known type of MDI and would have been obvious substitution of one known type of inhaler for another and to provide feedback to the user on how much, or little, synchronization occurs between actuation and inhalation (Bonney col. 3 ln. 16-25). The now modified Hanina method is silent as to the calculating of video and audio signals being done from a plurality of video frames and plurality of audio segments; however, Huang teaches a similar method which includes frame-by-frame analysis (see Huang abstract, para. 0020-0022, and 0033-0034). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the modified Hanina method to perform calculations on the video and audio signal based on frame-by-frame or segment-by-segment analysis, as taught by Huang, in order to provide a more in depth of analysis of the video and audio signal (analysis of individual plural frames/segments instead of merely the overall signal).
Regarding claim 5, the modified Hanina method’s calculating pressing probability for a video frame includes detecting in the video frame points representing the actuating finger and determining relative descending movement of the tip of the finger relative to a base compared to a preceding video frame and the pressing probability being a function of the amplitude of the descending movement from a starting position (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement, i.e. descending movement; it being obvious to track plural points of the finger including the tip based on use by the user).
Regarding claim 6, the modified Hanina method’s calculating of pressing probability includes comparing the amplitude of the movement to a dimension of the MDI (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement; the method requiring standard dimension/actuation length of the particular MDI in order to determine proper actuation).
Regarding claim 7, the modified Hanina method’s calculating compression probability includes comparing a length of the MDI with a reference length of the MDI (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement; the method requiring standard dimension/length of the particular MDI in order to determine proper actuation).
Regarding claim 9, the modified Hanina method’s step of calculating pressure/compression and inhalation probabilities are triggered by detection of proper positioning of the MDI relative to the patient in earlier video frames (see Hanina para. 0021, 0023, 0042, 0048, 0052, 0059, 0065 for example).
Regarding claim 19, the modified Hanina method’s calculating compression probability includes comparing a length of the MDI with a reference length of the MDI (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement; the method requiring standard dimension/length of the particular MDI in order to determine proper actuation).
Regarding claim 20, the modified Hanina method’s calculating compression probability includes comparing a length of the MDI with a reference length of the MDI (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement; the method requiring standard dimension/length of the particular MDI in order to determine proper actuation).
Regarding claim 11, Hanina discloses a computer system including one or more processors (see abstract, para. 0002, 0018-0024, 0038 for example) configured for obtaining a video signal and an audio signal of the patient using the MDI (see para. 0007, 0014, 0046, 0066 regarding MDI, para. 0022, 0038, 0041 regarding obtaining video and audio signals of a user using the inhaler); calculating from the video signal at least one of a so-called pressing probability that an actuating finger of the patient is actuating the MDI and a so-called compression probability that the MDI is in a actuated (see para. 0023 which discloses tracking/determining of movements associated with actuation of the inhaler including finger actuation which is indicative of MDI actuation, see also para. 0041, 0048, 0056-0057, 0059; see para. 0056, 0070, and 0080 which discloses determining levels of confidence of proper actuation which is a probability of actuation), calculating from the audio signal a so-called inhalation probability of the patient performing, in the audio signal, an inspiration combined with the aerosol stream (see para. 0023 which discloses calculating patient use/inhalation of the inhaler from audio signal, see also 0056 and 0066; see para. 0056, 0070, and 0080 which discloses determining levels of confidence of proper use including probability of inhalation), determining synchronization between actuation of the inhaler and an inspiration by the patient from the pressing/compression and halation probabilities to the same instants in time (see para. 0042, 0022-0023, 0041, see also abstract and para. 0002 which discloses analyzing to determine proper administration which includes synchronization of actuation and inhalation), and accordingly issuing to the patient a signal of proper use or misuse of the inhaler (see para. 0024, 0045). Hanina is silent as to the actuation of the MDI being finger pressing on a trigger member and/or compressed state of the MDI and to determine a degree of synchronization between inhalation and actuation; however, Bonney teaches a similar system/method which includes these limitations (see Bonney col. 3 ln. 16-25 regarding determining degree of synchronization between actuation and inhalation, col. 6 ln. 35-41 regarding finger pressing/compression of the MDI for actuation of an MDI). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the Hanina system to explicitly determine a degree of synchronization between inhalation and actuation and for the MDI to be of the finger pressing/compression state type MDI, as taught by Bonney, as this is a well-known type of MDI and would have been obvious substitution of one known type of inhaler for another and to provide feedback to the user on how much, or little, synchronization occurs between actuation and inhalation (Bonney col. 3 ln. 16-25). The now modified Hanina system is silent as to the calculating of video and audio signals being done from a plurality of video frames and plurality of audio segments; however, Huang teaches a similar method which includes frame-by-frame analysis (see Huang abstract, para. 0020-0022, and 0033-0034). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the modified Hanina system to perform calculations on the video and audio signal based on frame-by-frame or segment-by-segment analysis, as taught by Huang, in order to provide a more in depth of analysis of the video and audio signal (analysis of individual plural frames/segments instead of merely the overall signal).
Claim(s) 2, 13, 16, 3-4, 15, 18, 14, 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hanina, Bonney, and Huang as applied to claim 1 above, and further in view of Jung et al. (EP 3,485,930).
Regarding claim 2, the modified Hanina method is silent as to the degree of synchronization including providing temporal windows of probability including as a function of a temporal overlap between windows of probability; however, Jung teaches a similar method including temporal analysis of probabilities of actuation and synchronization of use of an inhaler (see Jung abstract and para. 0020, 0045). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the modified Hanina method to include analysis of temporal windows/overlap, as taught by Jung, in order to provide more in depth analysis of actuation vs inhalation synchronization (based on temporal analysis of Jung).
Regarding claim 13, the modified Hanina method’s calculating pressing probability for a video frame includes detecting in the video frame points representing the actuating finger and determining relative descending movement of the tip of the finger relative to a base compared to a preceding video frame and the pressing probability being a function of the amplitude of the descending movement from a starting position (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement, i.e. descending movement; it being obvious to track plural points of the finger including the tip based on use by the user).
Regarding claim 16, the modified Hanina method’s calculating compression probability includes comparing a length of the MDI with a reference length of the MDI (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement; the method requiring standard dimension/length of the particular MDI in order to determine proper actuation).
Regarding claim 3, the modified Hanina method is silent as to the degree of synchronization including combining for plurality of instants in time the probabilities of pressing/compression and inhalation to determine the degree of synchronization; however, Jung teaches a similar method including analysis of combined instants of time of probabilities of actuation and synchronization of use of an inhaler (see Jung abstract and para. 0020, 0045). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the modified Hanina method to include analysis of temporal windows/overlap, as taught by Jung, in order to provide more in depth analysis of actuation vs inhalation synchronization (based on temporal analysis of Jung).
Regarding claim 4, the modified Hanina method’s degree of synchronization includes comparing probabilities with a threshold value of proper synchronization (see Huang abstract, para. 0020-0021, 0032-0034; Jung para. 0046 regarding use of thresholds in analysis/determination).
Regarding claim 15, the modified Hanina method’s calculating pressing probability for a video frame includes detecting in the video frame points representing the actuating finger and determining relative descending movement of the tip of the finger relative to a base compared to a preceding video frame and the pressing probability being a function of the amplitude of the descending movement from a starting position (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement, i.e. descending movement; it being obvious to track plural points of the finger including the tip based on use by the user).
Regarding claim 18, the modified Hanina method’s calculating compression probability includes comparing a length of the MDI with a reference length of the MDI (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement; the method requiring standard dimension/length of the particular MDI in order to determine proper actuation).
Regarding claim 14, the modified Hanina method’s calculating pressing probability for a video frame includes detecting in the video frame points representing the actuating finger and determining relative descending movement of the tip of the finger relative to a base compared to a preceding video frame and the pressing probability being a function of the amplitude of the descending movement from a starting position (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement, i.e. descending movement; it being obvious to track plural points of the finger including the tip based on use by the user).
Regarding claim 17, the modified Hanina method’s calculating compression probability includes comparing a length of the MDI with a reference length of the MDI (see Hanina para. 0022-0023 which discloses motion tracking and the finger of the user being used to actuate the inhaler, see also Bonney col. 6 ln. 35-41 and figures regarding more specific actuation type MDI including compression movement; the method requiring standard dimension/length of the particular MDI in order to determine proper actuation).
Claim(s) 8 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hanina, Bonney, and Huang as applied to claim 1 above, and further in view of Reilly et al. (2015/0231343) and Haussermann et al. (2021/0110905).
Regarding claim 8, the modified Hanina method is silent as to converting the audio segment into a spectrogram and using it as input into a trained neural network to output inhalation probability; however, Reilly teaches a similar method including conversion of audio signal/segment into a spectrogram (see Reilly para. 0190) and Haussermann discloses a similar method including use of trained neural networks for an inhaler monitoring system (see Haussermann para. 0024-0032). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the modified Hanina method to convert audio segments into a spectrogram, as taught by Reilly, and using a trained neural network, as taught by Haussermann, in order to provide a more comprehensive analysis (via neural networks/AI) and known data formats (spectrograms).
Claim(s) 10 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hanina, Bonney, and Huang as applied to claim 1 above, and further in view of Kim (2023/0368686) and Parry-Billings et al. (2014/0158126).
Regarding claim 10, the modified Hanina method is silent as to further including an initial determining step of the opening of the MDI by detecting a characteristic click sound in at least one audio segment by a learnt detection model; however, Kim teaches a method for monitoring use of an inhaler including determining an initial opening of the inhaler via learnt detection model (see Kim para. 0084, as applied to Hanina process to be a learnt detection model, Hanina para. 0059, 0065-0066) and Parry-Billings teaches a known inhaler which produces an audible click sound when the cover is opened (see Parry-Billings para. 0123). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the modified Hanina method to determine an initial opening of the inhaler, as taught by Kim, via an audible click sound, taught by Parry-Billings, in order to provide an additional sequence of operation of use of the inhaler to be analyzed for compliance (the opening of the inhaler).
Claim(s) 12 is/are rejected under 35 U.S.C. 103 as being unpatentable over Hanina, Bonney, and Huang as applied to claim 1 above, and further in view of Haussermann.
Regarding claim 12, the modified Hanina method is a computer-readable medium storing a program executed by a microprocessor or computer system to carry out the method of claim 1 (see above discussion of the limitations of claim 1; Hanina para. 0002 and 0038 regarding the method performed by a computer system), but is silent as to the computer-readable medium explicitly being non-transient/non-transitory. However, Haussermann discloses a similar method which is implemented on a non-transient/non-transitory medium (see Haussermann para. 0076). Thus it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the modified Hanina method to be provided on a non-transient/non-transitory computer-readable medium, as taught by Haussermann, as this is a well-known type of computer-readable medium and would have been obvious to try or obvious substitution of one known element/type for another and one would expect the modified Hanina to perform equally as well.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure: Van Sickle et al. (2016/0256639) discloses a usage monitor for an inhaler, Shah (2015/0339953) discloses a self-learning system for an inhaler, Hillsman (4,984,158) discloses an inhaler biofeedback training/evaluation system, Bangera et al. (2014/0055589) discloses a computer monitoring inhaler system, and Schroeder et al. (9,084,864) discloses a metered dose device which is actuated by a user’s finger pressing down on a canister.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to COLIN W STUART whose telephone number is (571)270-7490. The examiner can normally be reached M-F: 9-5.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Justine Yu can be reached at 571-272-4835. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/COLIN W STUART/Primary Examiner, Art Unit 3785