Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-2, 4-10 is/are rejected under 35 U.S.C. 103 as being unpatentable over “NIU” [“Detection of sputum by interpreting the time-frequency distribution of respiratory sound signal using image processing techniques”], in view of “REN”, China Patent CN 114376600 and “EMMAN”, US Publication 20180317876.
Regarding claims 1 and 9, NIU discloses a system and associated method for determining a sputum type using a respiratory sound [abstract], the system comprising at least one processor configured to:
collect respiratory sound data from a patient who may have undergone tracheostomy [ section 2.1, 3.1];
receive the respiratory sound data collected by the at least one processor and convert the received respiratory sound data into a spectrogram image [section 2.2.2; equation 4]; and
determine a sputum type of the patient who may have undergone tracheostomy using machine learning classifiers based on a pattern difference between spectrogram images converted by the at least one processor [abstract; section 2.2.3].
Although NIU does not explicitly use the term “tracheostomy”, NIU’s approach of analyzing spectrogram patterns using machine learning to detect sputum is analogous in the context of patients needing assistance with sputum expectoration in NIU [e.g., ICU patients] and those undergoing tracheostomy, as both groups may have difficulty clearing sputum naturally. Accordingly, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to explicitly collect data from an ICU patient who has undergone tracheostomy in order to determine the sputum condition for appropriate action.
NIU discloses using machine learning classifiers, but did not explicitly disclose the use of a deep learning model for the analysis of the spectrogram images.
REN teaches a similar sputum analysis system that uses deep learning [convolutional neural network (CNN)] module for analyzing respiratory sound spectrum data which can be represented as spectrograms to determine sputum characteristics [abstract, “based on the convolutional neural network module sound spectrum for feature information extracting”].
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the teachings of NIU with the teachings of REN in order to enhance the base device of NIU with the well-known technique of deep learning. Specifically, REN’s explicit disclosure of using CNN for analyzing the spectral data [spectrograms] of respiratory sounds can be applied to NIU’s spectrogram generation process. An ordinary artisan would recognize that CNNs are particularly adept at image pattern recognition, making them a suitable and potentially more powerful tool than the classifiers used by NIU for analyzing the spectrogram images.
NIU classifies “sputum sound and non-sputum sound” (Data analysis, Classification method), broadly covering a “normal type” that would be expected to have smaller sound pressure than sputum types. NIU and REN combined does not explicitly disclose that the deep learning model is configured to classify the sputum type of the patient into a first sputum type that shows continuous and extensive sound pressure at a frequency of 2 kHz or more on the spectrogram image, and a second sputum type that shows repetitive vertical lines due to low-frequency vibration [although these patterns can be considered inherent].
EMMAN discloses classifications of respiratory sounds and their acoustic characteristics on spectrograms. EMMAN discusses “wheezes” which are “stationary in nature” and “have been reported to span a wide range of frequencies 100-2500 or 400-1600 Hz” [0058]. FIG. 6C shows a spectrogram of a wheeze, illustrating continuous sound pressure. EMMAN also states that “abnormal lung sounds… exhibit a frequency profile below 4 kHz” [0068], indicating that frequencies of 2 kHz or more are relevant for abnormal sound analysis. EMMAN further discusses “crackles” which are “transient and explosive” and “have been reported within various spectral ranges, below 2000 Hz, or above 500 Hz or between 100-500 Hz with a duration less than 20 msec with energy content in the lower frequency range of 100-500 Hz” [0058], [0059]. FIG. 6B shows a spectrogram of a crackle, which visually appears as short, vertical lines, consistent with “repetitive vertical lines due to low-frequency vibration.” Additionally, EMMAN shows the normal type having a smaller sound pressure than that of the other types as expected in FIG.6A.
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the explicit teachings of EMMAN into the system of NIU and REN. The incorporation of such natural occurrences would have been prompted in order to endow the system of NIU and REN with the capability to recognize such natural phenomenon. As such, one would have been motivated to apply EMMAN’s explicit acoustic descriptions to NIU and REN’s spectrogram-based machine learning to predictably result in a more granular and diagnostically useful classification of sputum types.
Regarding claim 2, NIU and REN combined discloses wherein the respiratory sound data collected by the at least one processor is classified into respiratory sound data requiring sputum suction and normal respiratory sound data [NIU: abstract; section 2.2.3; table 3].
Regarding claim 4, NIU and REN combined discloses wherein the at least one processor is configured to extract respiratory sound sample data corresponding to one breathing cycle from the respiratory sound data in which classification results of plurality of otolaryngologists are identical [NIU: section 2.2.1; fig.7; Results: “the presence of sputum was determined and categorized as either sputum or non-sputum” based on the “judgement of experienced nurses”]. EMMAN further teaches that annotation labels were made available for the full dataset by a panel of 8 reviewers and one expert reader” with “definite” labels requiring “at least two full breath cycles” [0066]. An ordinary artisan, when training a deep learning model for respiratory sound classification, would be motivated to use high-quality, accurately labeled data. Data labeled by experienced professionals with “identical” results represents such high-quality data. Furthermore, analyzing respiratory sounds often benefits from segmenting them into individual breathing cycles to capture specific events within those cycles. Combining these two known practices (using expert-validated data and analyzing breathing cycles) for training or refining a model would be a logical step to improve classification accuracy. NIU and EMMAN’s detailed description of expert annotation and arbitration further strengthens this.
Regarding claim 5, NIU and REN combined discloses wherein the at least one processor is configured to convert the respiratory sound data into the spectrogram image through short-time Fourier transform (STFT) [NIU: section 2.2.2; equations 2-4].
Regarding claim 6, NIU and REN combined discloses wherein the deep learning model is implemented as a convolution neural network (CNN) [REN: abstract].
Regarding claim 7, NIU and REN combined discloses wherein the at least one processor is configured to verify an accuracy of the deep learning model by using already classified respiratory sound data as input data [NIU: abstract; section 3 Results; applying leave-one-out cross validation with accuracy of 83.5%].
Regarding claim 8, NIU and REN combined discloses wherein the at least one processor is configured to verify the accuracy of the deep learning model using a predetermined performance evaluation index [NIU: section 3 Results; table 2-3; e.g., discrimination rate, sensitivity].
Regarding claim 10, NIU and REN combined discloses verifying an accuracy of the deep learning model by using already classified respiratory sound data as input data [NIU: section 3; cross-validation].
Response to Arguments
Applicant’s arguments submitted 4/6/26 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
In general, examiner submits that the claimed spectrogram features are the natural appearance of different sputum sounds – with classification of sputum/non-sputum, further subdividing based on spectral morphology would be a predictable extension. Therefore, once the art teaches spectrogram-based sputum detection, it would have been obvious, absent of any claimed specific classification algorithm, for deep learning networks such as CNNs to sort the sputum by visual texture.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
“KARAN”, WO 2019229543, discloses conversion of respiratory sound data into a spectrogram image and the use of a neural-network classifier on that image. It also teaches classifying respiratory sounds from spectrograms, including Mel spectrograms and color spectrograms. (KARAN, Summary: “the received sound record is transformed into a time-frequency domain graphical representation”; Introduction: “The time-frequency domain graphical representation includes a Mel spectrogram”; “the classifier model includes a neural network model”). KARAN provides explicit teaching on using “known sound classes determined by one or more experts” and “Groups of three physicians reached a consensus decision about the class of lung sound represented by each of the recorded sounds (sound records) of the data set” for training machine learning models (Description, Summary, Machine learning section).
“IKEDA”, WO 2013089073, discloses similar classifications of respiratory sounds related to secretions and their acoustic characteristics on spectrograms. IKEDA teaches classifying “continuous noise” caused by “accumulation of secretions in the trachea” and “intermittent noise” caused by “liquid secretions in the trachea” (Description, Functional configuration). IKEDA further describes “high-pitched continuous noise” where “frequency of the sound that sounds at a narrow airway… becomes high” (Description, Functional configuration), and illustrates continuous noise in FIG. 14, which shows continuous and extensive sound pressure. IKEDA also describes “intermittent noise” as a “plosive sound that occurs instantaneously” (Description, Functional configuration), and illustrates intermittent noise in FIG. 15, which visually depicts repetitive vertical lines -- this aligns with “repetitive vertical lines due to low-frequency vibration” (crackles are often low-frequency).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Tse Chen whose telephone number is (571)272-3672. The examiner can normally be reached M-F 7-3 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jonathan Moffat can be reached at 571-272-4390. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/TSE W CHEN/Supervisory Patent Examiner, Art Unit 3791
Prosecution Insights