DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Examiner Note Regarding Double Patenting
The pending claims of the Instant Application have been reviewed for double patenting in light of the allowed claims of U.S. Pat. No. 11,763,821 and U.S. Pat. No. 12,361,951. Though the pending claims do not currently constitute double patenting, examiner reserves the right to change this determination in light of later amendments provided by the applicant.
Response to Amendments
Applicant’s preliminary amendment filed on 24 March 2025 (hereinafter, Amendment) has been entered.
In view of the amendment to the claim(s), the amendment of claim(s) 1 and the addition of claim(s) 2-20 have been acknowledged and entered.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, as failing to comply with the written description requirement. The claim(s) contains subject matter which was not described in the specification in such a way as to reasonably convey to one skilled in the relevant art that the inventor or a joint inventor, or for applications subject to pre-AIA 35 U.S.C. 112, the inventor(s), at the time the application was filed, had possession of the claimed invention.
Regarding claim 1, and mutatis mutandis claims 7 and 14, the limitation “digitized sound information associated with one or both of a speech utterance by a speaker and an automated speech recognition (ASR) process” incorporates embodiments which are not taught by the specification as filed. Claim 1 recites “digitized sound information associated with one or both of a speech utterance by a speaker and an automated speech recognition (ASR) process” at line 2-3. For purposes of discussion, this limitation can be referred to as “ digitized sound information associated with one or both of A and B.” As understood by the office, this limitation captures 3 possible embodiments: “digitized sound information associated with… both of a speech utterance by a speaker and an automated speech recognition (ASR) process” (both A and B); “digitized sound information associated with… a speech utterance by a speaker” which is not associated with “an automated speech recognition (ASR) process” (only A); and “digitized sound information associated with… an automated speech recognition (ASR) process” which is not associated with “a speech utterance by a speaker” (only B). Respectfully, the “only A” embodiment and the “only B” embodiment are not clearly disclosed in the specification as filed such that one skilled in the art would recognize these limitations as part of the captured invention at the time of filing.
The presented limitations of claim 1 and new claims 2-20 were incorporated by the Amendment, filed after the filing date of the application. As such, the amendments are not entitled to treatment as part of the original disclosure of the application. (See MPEP 714.01(e) citing 37 CFR 1.115). In support of the preliminary amendment, applicant indicates that support can be found in the description, claims, and drawings of the application as originally filed.” (Amendment, pg. 7). However, upon review of the specification, including description, claims, and drawings, clear specification support for the “only A” embodiment or the “only B” embodiment could not be found.
Further, the alternative combinations result in further embodiments which are not clearly disclosed in the specification. For example, regarding the “only A” embodiment, the specification does not clearly disclose an operation corresponding to an ASR process performed on digitized sound information which is not associated with an ASR process. Regarding the “only B” embodiment, the specification does not clearly disclose a method of determining one or more top-N ASR recognized matches from digitized sound information which is not associated with a speech utterance by a speaker. As well, the specification fails to clearly disclose receiving a selection of a pictogram which approximates an intended meaning associated with the speech utterance based on digitized sound information which is not associated with a speech utterance. Therefore, each of claims 1, 7, and 14 contain at least one limitation which constitutes new matter and the claims are therefore rejected.
Regarding claims 2-6, 8-13, and 15-20, claims 2-6, 8-13, and 15-20 depend from claims 1, 7, and 14, and incorporate all limitations therefrom. Therefore, claims 2-6, 8-13, and 15-20 are rejected under 112(a) for at least the same reasons as claims 1, 7, and 14.
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Regarding claim 1, and mutatis mutandis claims 7 and 14, the limitation “performing an operation corresponding to the ASR process in association with the digitized sound information to determine: one or more top-N ASR-recognized matches between the digitized sound information and the past ASR-recognized utterances” lacks clarity. Claim 1 recites “performing an operation... to determine: one or more top-N ASR-recognized matches” at lines 6-8. However, it is unclear what forms the basis for the described “matches”. The ordinary definition of the word “match” requires a comparison between two objects. A first object becomes “one or more top-N ASR-recognized matches”, for a second object. The claim recites top-N ASR matches, but does not disclose an ASR process being performed for the digitized sound information, which also may not even include “a speech utterance of a speaker”, thus it is unclear what is being matched “between the digitized sound information and the past ASR-recognized utterances.” As such, claim 1 does not disclose or even necessarily include any corresponding first object as the matching target for the second object (which may be “past ASR-recognized utterances”, a corresponding waveform in a speech segment, etc.). As such, the operation recited in claim 1 is incomplete and lacks clarity. Therefore, claims 1, 7, and 14 are rejected as being indefinite under 112(b).
Regarding claims 2-6, 8-13, and 15-20, claims 2-6, 8-13, and 15-20 depend from claims 1, 7, and 14, and incorporate all limitations therefrom. Therefore, claims 2-6, 8-13, and 15-20 are rejected under 112(b) for at least the same reasons as claims 1, 7, and 14.
Appropriate correction is required.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-2, 7-8, and 14-15 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morris (U.S. Pat. App. Pub. No. 2018/0018144, hereinafter Morris) in view of Gao (U.S. Pat. App. Pub. No. 2004/0111272, hereinafter Gao) with further evidence from Non-Patent Literature to Blissymbolics Communication International (Blissymbolics Communication International, 2014. BCI Authorized Vocabulary (BCI-AV), <url: https://web.archive.org/web/20170214193806/ http://www.symbolnet.org/bliss/BCI-AV_2014-12-05_(en+deriv+es+sv)_symbols-draft-3.pdf>, [Pub. On. 15 Dec. 2014], [Retr. on: 05 Sep. 2026] (Year: 2014), hereinafter Bliss).
Regarding claim 1, Morris discloses A computer-implemented method (Systems and methods described with reference to “an example device 100 configured to operate according to the techniques discussed herein to enhance communication throughput by leveraging environmental data.”; Morris, ¶ [0045]), comprising: acquiring digitized sound information associated with one or both of a speech utterance by a speaker and an automated speech recognition (ASR) process (“ the context capture module 120 can configure the example device 100 to obtain and/or capture contextual data, according to any of the techniques discussed herein” where “the contextual data 204(1) can include… utterances of the user 206” and “utterance can include any communication method available to a user,” which includes speech utterances of the user; Morris, ¶ [0053]); receiving, via one or more hardware processors, past ASR-recognized utterances representative of the speaker (“the suggestion generator 122 can start with a set of heuristic phrases which the suggestion generator 122 can augment using machine learning (e.g., by a deep learning network, Naïve Bayes classifier, directed graph) using data regarding the suggestion selection activity and/or utterance patterns of one or more users (e.g., by accessing past utterances of a user, by accessing past utterances and/or selections of a multiplicity of users stored on a cloud service). “; Morris, ¶ [0054]); performing an operation corresponding to the ASR process in association with the digitized sound information to determine: one or more top-N ASR-recognized matches (“the suggestion generator 122” performs the process of “generate predicted words, identify words from which to generate other words and/or phrases, and/or generate predicted phrases,” corresponding to “an audio-to-text conversion service, a video-to-audio and/or image conversion service, a vision service, and/or a natural language processing service” which may comprise “N suggestions” which are selected and presented “according to the weighting, sorting, ranking, and/or filtering {one or more top-N ASR-recognized matches}”; Morris, ¶ [0054], [0091]) [one or more top-N ASR-recognized matches] between the digitized sound information and the past ASR-recognized utterances (As described above, “the suggestion generator 122” performs “audio-to-text conversion” using machine learning, the machine learning being trained in part on past utterances and/or selections of the user and “contextual data can include previous utterances received from application data”. As such, the recognition results and corresponding matches generated therefrom, are between the digitized sound information and the past ASR-recognized utterances.; Morris, ¶ [0054], [0090]); determining, via the one or more hardware processors, one or more pictograms corresponding to the one or more top-N ASR-recognized matches, wherein the one or more pictograms represent one or more words and/or concepts (“the generated words and/or phrases are collectively referred to herein as suggestions” which are “predictions (i.e., calculated anticipations) of words and/or phrases” which are generated in response to the contextual data and said generated words can be “pictographic representations”, which, as a “representation of the one or more suggested utterances” is further is a determination of the same pictorial representations corresponding to the predictions.; Morris, ¶ [0043], [0123]); causing presentation, via an electronic user interface associated with the one or more hardware processors, of the one or more pictograms (Discloses “the outputting including: one or more of: rendering the one or more suggested utterances via the human-machine interface as a symbol, word, or phrase completion” and “rendering a pictorial representation of the one or more suggested utterances via the human-machine interface {electronic interface associated with the one or more hardware processors}”; Morris, ¶ [0123]); and receiving, via the one or more hardware processors and in response to the presentation, an indication of a selected pictogram from the one or more pictograms, (Further discloses “receiving an input indicative of a selection of a word or a phrase of the one or more suggested utterances” where the word or phrase selected is the “pictographic representation”; Morris, ¶ [0043], [0123], [0125]). However, Morris fails to expressly recite wherein the selected pictogram approximates an intended meaning associated with the speech utterance.
Gao teaches systems and methods “for translating a natural language sentence of a source language into a symbolic representation.” (Gao, ¶ [0009]). Regarding claim 1, Gao teaches wherein the selected pictogram approximates an intended meaning associated with the speech utterance (“The symbolic image generator 130 may access image symbolic models, e.g., Blissymbolics or Minspeak, to generate the symbolic representation. Here, the generator 130 will extract the appropriate symbols to create “words” to represent different elements of the original source sentence and group the “words” together to convey an intended meaning of the original source sentence. “; Gao, ¶ [0039]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris to incorporate the teachings of Gao to include wherein the selected pictogram approximates an intended meaning associated with the speech utterance. The symbolic representation translation system of Gao can apply “natural language understanding technology to classify concepts and semantics in a spoken sentence” and “translate the sentence into a [symbolic representation]” to “show the main concepts and semantics in the sentence” which aids in comprehension, especially with speech impairments, such as “the deaf” or for people who “are illiterate.” (Gao, ¶ [0007]-[0009]).
Regarding claim 2, the rejection of claim 1 is incorporated. Morris discloses all of the elements of the current invention as stated above. However, Morris fail(s) to expressly recite further comprising identifying an indicia relating to each of the one or more top-N ASR-recognized matches, wherein the indicia indicates a word associated with a particular ASR-recognized match of the one or more ASR- recognized matches, a concept associated with the particular ASR-recognized match, a word/concept pictogram of the one or more word/concept pictograms, or any combination thereof.
The relevance of Gao is described above with relation to claim 1. Regarding claim 2, Gao teaches further comprising identifying an indicia relating to each of the one or more top-N ASR-recognized matches, (“The interlingua is also sent to the symbolic image generator 130 for generating a symbolic representation of visual depictions to be displayed on image display 106 (step 214). The symbolic image generator 130 may access image symbolic models, e.g., Blissymbolics or Minspeak, to generate the symbolic representation” The indicia indications are not expressly recited in Gao. However, in disclosing the Blissymbolics and the Minspeak models, Gao further includes the necessary components which define those models. The necessary and defining components which are disclosed by the reference to each of said symbolic models, include that Blissymbolics includes a Bliss Identification Number (BCI-AV numbers, See Bliss at col. 1, throughout), Compositional and semantic encoding strings (Bliss-Words/Bliss Characters, See Bliss at col. 2, throughout), and grammatical category flags (Indicators, See Bliss, at least at pages 8-10). Though, not considered necessary for the rejection, Minspeak also includes Icon Addresses/Key IDs, sequence vectors/transition keys, and Semantic Compaction codes, which are indicia relating to the top-N ASR recognized matches. These components of the Blissymbolics and Minspeak models would be known to a person having ordinary skill in the art and understood to be incorporated into the disclosure of Gao.; Gao, ¶ [0039]) wherein the indicia indicates a word associated with a particular ASR-recognized match of the one or more ASR- recognized matches, a concept associated with the particular ASR-recognized match, a word/concept pictogram of the one or more word/concept pictograms, or any combination thereof (In the Blissymbolics example, as used in Gao, BCI-AV numbers map directly to a specific target text string in the system’s dictionary (as shown in Bliss, throughout), where the Bliss-Words (the indicia) represents the abstract concept itself as defined by the combination of base primitives (also referred to as Bliss-characters), and indicates a pictogram as the BCI-AV numbers function as the database primary key/file path/rendering pointer used by the GUI to retrieve and display the geometric Bliss-Words/Bliss-characters on the display. Once more, though not considered necessary for the rejection, in the Minspeak example, where the indicia is the icon sequence vector/semantic compaction code, which operates as a lookup key, mapping to the stored target word or phrase in the vocabulary database (e.g., in one example, the sequence vector for [Apple]+[Verb] resolves to the text string [Eat]), and further captures the conceptual relationship used to disambiguate polysemy encoding the semantic rule being applied, and the individual icon IDs that make up the sequence vector are the graphical pointers used to render the pictograms on the display.; Gao, ¶ [0039]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris to incorporate the teachings of Gao to include further comprising identifying an indicia relating to each of the one or more top-N ASR-recognized matches, wherein the indicia indicates a word associated with a particular ASR-recognized match of the one or more ASR- recognized matches, a concept associated with the particular ASR-recognized match, a word/concept pictogram of the one or more word/concept pictograms, or any combination thereof. The symbolic representation translation system of Gao can apply “natural language understanding technology to classify concepts and semantics in a spoken sentence” and “translate the sentence into a [symbolic representation]” to “show the main concepts and semantics in the sentence” which aids in comprehension, especially with speech impairments, such as “the deaf” or for people who “are illiterate.” (Gao, ¶ [0007]-[0009]).
Regarding claim 7, Morris discloses One or more non-transitory media having instructions that, when executed by one or more hardware processors, cause the one or more hardware processors to facilitate a plurality of operations, the operations comprising (Systems and methods described with reference to “an example device 100 configured to operate according to the techniques discussed herein to enhance communication throughput by leveraging environmental data,” where the device 100 “can include any type of computing device having one or more processing unit(s) 110 operably connected to computer-readable media 112.” and the “computer-readable media 112 can store instructions readable and/or executable by the processing unit(s) 110. Executable instructions stored on computer-readable media 112 can include, for example, the ECTF 108 and other modules, programs, and/or applications that can be loadable and executable by processing unit(s) 110 (e.g., operating system 116, application(s) 118, firmware).”; Morris, ¶ [0045], [0047], [0050], [0052]): acquiring digitized sound information associated with one or both of a speech utterance by a speaker and an automated speech recognition (ASR) process (“ the context capture module 120 can configure the example device 100 to obtain and/or capture contextual data, according to any of the techniques discussed herein” where “the contextual data 204(1) can include… utterances of the user 206” and “utterance can include any communication method available to a user,” which includes speech utterances of the user; Morris, ¶ [0053]); receiving, via the one or more hardware processors, past ASR-recognized utterances representative of the speaker (“the suggestion generator 122 can start with a set of heuristic phrases which the suggestion generator 122 can augment using machine learning (e.g., by a deep learning network, Naïve Bayes classifier, directed graph) using data regarding the suggestion selection activity and/or utterance patterns of one or more users (e.g., by accessing past utterances of a user, by accessing past utterances and/or selections of a multiplicity of users stored on a cloud service). “; Morris, ¶ [0054]); performing an operation corresponding to the ASR process in association with the digitized sound information to determine: one or more top-N ASR-recognized matches (“the suggestion generator 122” performs the process of “generate predicted words, identify words from which to generate other words and/or phrases, and/or generate predicted phrases,” corresponding to “an audio-to-text conversion service, a video-to-audio and/or image conversion service, a vision service, and/or a natural language processing service” which may comprise “N suggestions” which are selected and presented “according to the weighting, sorting, ranking, and/or filtering {one or more top-N ASR-recognized matches}”; Morris, ¶ [0054], [0091]) [one or more top-N ASR-recognized matches] between the digitized sound information and the past ASR-recognized utterances (As described above, “the suggestion generator 122” performs “audio-to-text conversion” using machine learning, the machine learning being trained in part on past utterances and/or selections of the user and “contextual data can include previous utterances received from application data”. As such, the recognition results and corresponding matches generated therefrom, are between the digitized sound information and the past ASR-recognized utterances.; Morris, ¶ [0054], [0090]); determining, via the one or more hardware processors, one or more pictograms corresponding to the one or more top-N ASR-recognized matches, wherein the one or more pictograms represent one or more words and/or concepts (“the generated words and/or phrases are collectively referred to herein as suggestions” which are “predictions (i.e., calculated anticipations) of words and/or phrases” which are generated in response to the contextual data and said generated words can be “pictographic representations”, which, as a “representation of the one or more suggested utterances” is further is a determination of the same pictorial representations corresponding to the predictions.; Morris, ¶ [0043], [0123]); causing presentation, via an electronic user interface associated with the one or more hardware processors, of the one or more pictograms (Discloses “the outputting including: one or more of: rendering the one or more suggested utterances via the human-machine interface as a symbol, word, or phrase completion” and “rendering a pictorial representation of the one or more suggested utterances via the human-machine interface {electronic interface associated with the one or more hardware processors}”; Morris, ¶ [0123]); and receiving, via the one or more hardware processors and in response to the presentation, an indication of a selected pictogram from the one or more pictograms, (Further discloses “receiving an input indicative of a selection of a word or a phrase of the one or more suggested utterances” where the word or phrase selected is the “pictographic representation”; Morris, ¶ [0043], [0123], [0125]). However, Morris fails to expressly recite wherein the selected pictogram approximates an intended meaning associated with the speech utterance.
The relevance of Gao is described above with relation to claim 1. Regarding claim 7, Gao teaches wherein the selected pictogram approximates an intended meaning associated with the speech utterance (“The symbolic image generator 130 may access image symbolic models, e.g., Blissymbolics or Minspeak, to generate the symbolic representation. Here, the generator 130 will extract the appropriate symbols to create “words” to represent different elements of the original source sentence and group the “words” together to convey an intended meaning of the original source sentence. “; Gao, ¶ [0039]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris to incorporate the teachings of Gao to include wherein the selected pictogram approximates an intended meaning associated with the speech utterance. The symbolic representation translation system of Gao can apply “natural language understanding technology to classify concepts and semantics in a spoken sentence” and “translate the sentence into a [symbolic representation]” to “show the main concepts and semantics in the sentence” which aids in comprehension, especially with speech impairments, such as “the deaf” or for people who “are illiterate.” (Gao, ¶ [0007]-[0009]).
Regarding claim 8, the rejection of claim 7 is incorporated. Claim 8 is substantially the same as claim 2 and is therefore rejected under the same rationale as above.
Regarding claim 14, Morris discloses A system having one or more hardware processors configured to facilitate a plurality of operations, the operations comprising (Systems and methods described with reference to “an example device 100 configured to operate according to the techniques discussed herein to enhance communication throughput by leveraging environmental data,” where the device 100 “can include any type of computing device having one or more processing unit(s) 110 operably connected to computer-readable media 112.” and the “computer-readable media 112 can store instructions readable and/or executable by the processing unit(s) 110. Executable instructions stored on computer-readable media 112 can include, for example, the ECTF 108 and other modules, programs, and/or applications that can be loadable and executable by processing unit(s) 110 (e.g., operating system 116, application(s) 118, firmware).”; Morris, ¶ [0045], [0047], [0050], [0052]): acquiring digitized sound information associated with one or both of a speech utterance by a speaker and an automated speech recognition (ASR) process (“ the context capture module 120 can configure the example device 100 to obtain and/or capture contextual data, according to any of the techniques discussed herein” where “the contextual data 204(1) can include… utterances of the user 206” and “utterance can include any communication method available to a user,” which includes speech utterances of the user; Morris, ¶ [0053]); receiving, via the one or more hardware processors, past ASR-recognized utterances representative of the speaker (“the suggestion generator 122 can start with a set of heuristic phrases which the suggestion generator 122 can augment using machine learning (e.g., by a deep learning network, Naïve Bayes classifier, directed graph) using data regarding the suggestion selection activity and/or utterance patterns of one or more users (e.g., by accessing past utterances of a user, by accessing past utterances and/or selections of a multiplicity of users stored on a cloud service). “; Morris, ¶ [0054]); performing an operation corresponding to the ASR process in association with the digitized sound information to determine: one or more top-N ASR-recognized matches (“the suggestion generator 122” performs the process of “generate predicted words, identify words from which to generate other words and/or phrases, and/or generate predicted phrases,” corresponding to “an audio-to-text conversion service, a video-to-audio and/or image conversion service, a vision service, and/or a natural language processing service” which may comprise “N suggestions” which are selected and presented “according to the weighting, sorting, ranking, and/or filtering {one or more top-N ASR-recognized matches}”; Morris, ¶ [0054], [0091]) [one or more top-N ASR-recognized matches] between the digitized sound information and the past ASR-recognized utterances (As described above, “the suggestion generator 122” performs “audio-to-text conversion” using machine learning, the machine learning being trained in part on past utterances and/or selections of the user and “contextual data can include previous utterances received from application data”. As such, the recognition results and corresponding matches generated therefrom, are between the digitized sound information and the past ASR-recognized utterances.; Morris, ¶ [0054], [0090]); determining, via the one or more hardware processors, one or more pictograms corresponding to the one or more top-N ASR-recognized matches, wherein the one or more pictograms represent one or more words and/or concepts (“the generated words and/or phrases are collectively referred to herein as suggestions” which are “predictions (i.e., calculated anticipations) of words and/or phrases” which are generated in response to the contextual data and said generated words can be “pictographic representations”, which, as a “representation of the one or more suggested utterances” is further is a determination of the same pictorial representations corresponding to the predictions.; Morris, ¶ [0043], [0123]); causing presentation, via an electronic user interface associated with the one or more hardware processors, of the one or more pictograms (Discloses “the outputting including: one or more of: rendering the one or more suggested utterances via the human-machine interface as a symbol, word, or phrase completion” and “rendering a pictorial representation of the one or more suggested utterances via the human-machine interface {electronic interface associated with the one or more hardware processors}”; Morris, ¶ [0123]); and receiving, via the one or more hardware processors and in response to the presentation, an indication of a selected pictogram from the one or more pictograms, (Further discloses “receiving an input indicative of a selection of a word or a phrase of the one or more suggested utterances” where the word or phrase selected is the “pictographic representation”; Morris, ¶ [0043], [0123], [0125]). However, Morris fails to expressly recite wherein the selected pictogram approximates an intended meaning associated with the speech utterance.
The relevance of Gao is described above with relation to claim 1. Regarding claim 14, Gao teaches wherein the selected pictogram approximates an intended meaning associated with the speech utterance (“The symbolic image generator 130 may access image symbolic models, e.g., Blissymbolics or Minspeak, to generate the symbolic representation. Here, the generator 130 will extract the appropriate symbols to create “words” to represent different elements of the original source sentence and group the “words” together to convey an intended meaning of the original source sentence. “; Gao, ¶ [0039]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris to incorporate the teachings of Gao to include wherein the selected pictogram approximates an intended meaning associated with the speech utterance. The symbolic representation translation system of Gao can apply “natural language understanding technology to classify concepts and semantics in a spoken sentence” and “translate the sentence into a [symbolic representation]” to “show the main concepts and semantics in the sentence” which aids in comprehension, especially with speech impairments, such as “the deaf” or for people who “are illiterate.” (Gao, ¶ [0007]-[0009]).
Regarding claim 15, the rejection of claim 14 is incorporated. Claim 15 is substantially the same as claim 2 and is therefore rejected under the same rationale as above.
Claims 3, 5-6, 9, 11-12, 16, and 18-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morris and Gao as applied to claim 1-2, 7-8, and 14-15 above, and further in view of Tang (U.S. Pat. App. Pub. No. 2016/0188573, hereinafter Tang).
Regarding claim 3, the rejection of claim 2 is incorporated. Morris and Gao disclose all of the elements of the current invention as stated above. However, Morris and Gao fail to expressly recite further comprising configuring a finite state machine (FSM) model to determine a set of information corresponding at least partially to the indicia or the one or more ASR-recognized matches.
Tang teaches systems and methods for “integrating domain information into state transitions of a Finite State Transducer for natural language processing.” (Tang, ¶ [0002]). Regarding claim 3, Tang teaches further comprising configuring a finite state machine (FSM) model to determine a set of information corresponding at least partially to the indicia or the one or more ASR-recognized matches (“a system 100 of integrating domain information into state transitions of a Finite State Transducer for natural language processing, according to an implementation of the invention. A system may integrate semantic parsing and information retrieval from an information domain to generate an FST parser that represents the information domain. The FST parser may include a plurality of FST paths, at least one of which may be used to generate a meaning representation from a natural language input (e.g., natural language utterance and/or other natural language input),” where the meaning of a natural language utterance corresponds to both the indicia and to the ASR-recognized matches.; Tang, ¶ [0047]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris, as modified by the symbolic representation systems of Gao, to incorporate the teachings of Tang to include further comprising configuring a finite state machine (FSM) model to determine a set of information corresponding at least partially to the indicia or the one or more ASR-recognized matches. Morris and Gao disclose an AAC device which receives contextual information including a user input and produces a plurality of symbolic representations such that the user can select the intended meaning. However, Morris and Gao are silent regarding the use of a finite state machine. Tang discloses integrating “semantic parsing and information retrieval from an information domain to generate an FST parser that represents the information domain… which may be used to generate a meaning representation from a natural language input (e.g., a natural language string based on a natural language utterance or other input)” by “perform[ing] domain-based semantic parsing of a natural language input to generate robust meaning representations (e.g., search queries, commands, etc.) using domain information” which provides the known benefit of “generating more relevant meaning representations,” which in the context of Morris and Gao would result in better accuracy in the text to pictogram mapping, reflecting meaning representations which are “more accurately aligned with the user's intent,” as recognized in the context of Tang. (Tang, [0007])
Regarding claim 5, the rejection of claim 1 is incorporated. Morris and Gao disclose all of the elements of the current invention as stated above. However, Morris and Gao fail to expressly recite further comprising initiating a finite state machine (FSM) model to determine a set of information corresponding at least partially to at least one of (i) indicia relating to each of the one or more top-N ASR-recognized matches or (ii) the one or more word/concept pictograms.
The relevance of Tang is described above with relation to claim 3. Regarding claim 5, Tang teaches further comprising initiating a finite state machine (FSM) model to determine a set of information corresponding at least partially to at least one of (i) indicia relating to each of the one or more top-N ASR-recognized matches or (ii) the one or more word/concept pictograms (“a system 100 of integrating domain information into state transitions of a Finite State Transducer for natural language processing, according to an implementation of the invention. A system may integrate semantic parsing and information retrieval from an information domain to generate an FST parser that represents the information domain. The FST parser may include a plurality of FST paths, at least one of which may be used to generate a meaning representation from a natural language input (e.g., natural language utterance and/or other natural language input),” where the meaning of a natural language utterance corresponds to both the indicia and to the ASR-recognized matches.; Tang, ¶ [0047]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris, as modified by the symbolic representation systems of Gao, to incorporate the teachings of Tang to include further comprising initiating a finite state machine (FSM) model to determine a set of information corresponding at least partially to at least one of (i) indicia relating to each of the one or more top-N ASR-recognized matches or (ii) the one or more word/concept pictograms. Morris and Gao disclose an AAC device which receives contextual information including a user input and produces a plurality of symbolic representations such that the user can select the intended meaning. However, Morris and Gao are silent regarding the use of a finite state machine. Tang discloses integrating “semantic parsing and information retrieval from an information domain to generate an FST parser that represents the information domain… which may be used to generate a meaning representation from a natural language input (e.g., a natural language string based on a natural language utterance or other input)” by “perform[ing] domain-based semantic parsing of a natural language input to generate robust meaning representations (e.g., search queries, commands, etc.) using domain information” which provides the known benefit of “generating more relevant meaning representations,” which in the context of Morris and Gao would result in better accuracy in the text to pictogram mapping, reflecting meaning representations which are “more accurately aligned with the user's intent,” as recognized in the context of Tang. (Tang, [0007])
Regarding claim 6, the rejection of claim 5 is incorporated. Morris, Gao, and Tang disclose all of the elements of the current invention as stated above. However, Morris and Gao fail to expressly recite further comprising retrieving information corresponding to the FSM model to determine a set of candidate words or phrases.
The relevance of Tang is described above with relation to claim 3. Regarding claim 6, Tang teaches further comprising retrieving information corresponding to the FSM model to determine a set of candidate words or phrases (“In an operation 204, FST parser generator 130 may integrate information from an information domain 150 to generate semantically structured FST paths based on tokens (e.g., words) from the information domain. For example, FST parser generator 130 may add state transitions to a semantically structured FST path based on tokens from an information domain.”; Tang, ¶ [0064]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris, as modified by the symbolic representation systems of Gao, to incorporate the teachings of Tang to include further comprising retrieving information corresponding to the FSM model to determine a set of candidate words or phrases. Morris and Gao disclose an AAC device which receives contextual information including a user input and produces a plurality of symbolic representations such that the user can select the intended meaning. However, Morris and Gao are silent regarding the use of a finite state machine. Tang discloses integrating “semantic parsing and information retrieval from an information domain to generate an FST parser that represents the information domain… which may be used to generate a meaning representation from a natural language input (e.g., a natural language string based on a natural language utterance or other input)” by “perform[ing] domain-based semantic parsing of a natural language input to generate robust meaning representations (e.g., search queries, commands, etc.) using domain information” which provides the known benefit of “generating more relevant meaning representations,” which in the context of Morris and Gao would result in better accuracy in the text to pictogram mapping, reflecting meaning representations which are “more accurately aligned with the user's intent,” as recognized in the context of Tang. (Tang, [0007])
Regarding claim 9, the rejection of claim 8 is incorporated. Claim 9 is substantially the same as claim 3 and is therefore rejected under the same rationale as above.
Regarding claim 11, the rejection of claim 7 is incorporated. Claim 11 is substantially the same as claim 5 and is therefore rejected under the same rationale as above.
Regarding claim 12, the rejection of claim 11 is incorporated. Claim 12 is substantially the same as claim 6 and is therefore rejected under the same rationale as above.
Regarding claim 16, the rejection of claim 15 is incorporated. Claim 16 is substantially the same as claim 3 and is therefore rejected under the same rationale as above.
Regarding claim 18, the rejection of claim 14 is incorporated. Claim 18 is substantially the same as claim 5 and is therefore rejected under the same rationale as above.
Regarding claim 19, the rejection of claim 18 is incorporated. Claim 19 is substantially the same as claim 6 and is therefore rejected under the same rationale as above.
Claims 4, 10, and 17 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morris, Gao, and Tang as applied to claim 3, 9, and 16 above, and further in view of Non-patent Literature to Niparnan (Niparnan, N. and Chongstitvatana, P., 2002, October. An improved genetic algorithm for the inference of finite state machine. In IEEE International Conference on Systems, Man and Cybernetics (Vol. 7, pp. 5-pp). IEEE., hereinafter Niparnan).
Regarding claim 4, the rejection of claim 3 is incorporated. Morris, Gao, and Tang disclose all of the elements of the current invention as stated above. However, Morris, Gao, and Tang fail to expressly recite wherein the FSM model is determined using a genetic or evolutionary algorithm.
Niparnan teaches a “method of the use of genetic algorithms to synthesize a finite state machine consistent with a given input/output sequence set.” (Niparnan, Abstract). Regarding claim 4, Niparnan teaches wherein the FSM model is determined using a genetic or evolutionary algorithm (Discloses the use of both genetic algorithms and evolutionary algorithms for the inference (determination) of finite state automata (machines) “consistent with given input/output sequences”; Niparnan, ¶ Page 1, paragraphs 2, 7, and 8).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris, as modified by the symbolic representation systems of Gao, and as modified by the natural language processing systems of Tang, to incorporate the teachings of Niparnan to include wherein the FSM model is determined using a genetic or evolutionary algorithm. The system described in Niparnan allows for synthesis of a finite state machine using input/output sequences which is “consistent with [observed] partial input/output sequences” while “reduc[ing] the search space in the problem and guid[ing] the search process in a more accurate direction.” (Niparnan, Page 1, paragraphs 1; Page 5, paragraph 2).
Regarding claim 10, the rejection of claim 9 is incorporated. Claim 10 is substantially the same as claim 4 and is therefore rejected under the same rationale as above.
Regarding claim 17, the rejection of claim 16 is incorporated. Claim 17 is substantially the same as claim 4 and is therefore rejected under the same rationale as above.
Claims 13 and 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Morris and Gao as applied to claim 1-2, 7-8, and 14-15 above, and further in view of Potter (U.S. Pat. App. Pub. No. 2008/0319750, hereinafter Potter).
Regarding claim 13, the rejection of claim 7 is incorporated. Morris and Gao disclose all of the elements of the current invention as stated above. However, Morris fails to expressly recite wherein the operations further comprise determining a match figure-of-merit for each of the one or more top-N ASR- recognized matches.
The relevance of Gao is described above with relation to claim 1. Regarding claim 13, Gao teaches wherein the operations further comprise determining a match figure-of-merit for each of the one or more top-N ASR- recognized matches (“The symbolic image generator 130 may access image symbolic models, e.g., Blissymbolics or Minspeak, to generate the symbolic representation. Here, the generator 130 will extract the appropriate symbols to create “words” to represent different elements of the original source sentence and group the “words” together to convey an intended meaning of the original source sentence. “ As read in the context of the “N suggestions”, this results in a specific symbolic representation {figure of merit} for each of the suggestions in the “N suggestions”; Gao, ¶ [0039]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris to incorporate the teachings of Gao to include wherein the operations further comprise determining a match figure-of-merit for each of the one or more top-N ASR- recognized matches. The symbolic representation translation system of Gao can apply “natural language understanding technology to classify concepts and semantics in a spoken sentence” and “translate the sentence into a [symbolic representation]” to “show the main concepts and semantics in the sentence” which aids in comprehension, especially with speech impairments, such as “the deaf” or for people who “are illiterate.” (Gao, ¶ [0007]-[0009]). However, Morris and Gao fail to expressly recite wherein the figure-of-merit includes a numerical confidence or probability having a value between 0 and 1, and wherein the numerical confidence or probability represents a closeness of matching or likelihood that an ASR-matched word or phrase correctly represents the intended meaning associated with the speech utterance.
Potter teaches systems and methods for “monitoring a spoken-word audio stream for a relevant concept.” (Potter, ¶ [0007]). Regarding claim 13, Potter teaches wherein the figure-of-merit includes a numerical confidence or probability having a value between 0 and 1 (“the speech recognition engine 200 may determine a confidence score 212 for each word of the plurality of words” where a confidence score is a numerical confidence or probability, and where “the confidence score 212 may be a number between zero and one.”; Potter, ¶ [0029], [0055]), and wherein the numerical confidence or probability represents a closeness of matching or likelihood that an ASR-matched word or phrase correctly represents the intended meaning associated with the speech utterance (“The confidence score 212 may include a number associated with the likelihood that the recognized instance 210 correctly matches the spoken word and/or phrase from the audio stream 208.”; Potter, ¶ [0029]).
It would have been prima facie obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the environmental context AAC device of Morris, as modified by the symbolic representation systems of Gao, to incorporate the teachings of Potter to include wherein the figure-of-merit includes a numerical confidence or probability having a value between 0 and 1, and wherein the numerical confidence or probability represents a closeness of matching or likelihood that an ASR-matched word or phrase correctly represents the intended meaning associated with the speech utterance. Morris and Gao disclose an AAC device which receives contextual information including a user input and produces a plurality of symbolic representations such that the user can select the intended meaning. However, Morris and Gao fail to expressly recite the use of a confidence value associated with the symbolic representation and the ASR hypothesis. Potter discloses the use of confidence values as part of the spoken word monitoring system, which allows for “effective identification of topics and their temporal location within a spoken-word audio stream.” As established in Potter confidence values for an ASR hypothesis, as used in real-time audio processing, were a quantifiable measure of accuracy {a known solution}, which could be readily applied to the ASR systems of Morris and Gao {a known system ready for improvement}, to enable a system to rank, filter, or prioritize candidate matches, from a set of ASR hypotheses for a speech utterance, based on their likelihood of being correct, to yield the predictable result of a more robust and accurate translation of an ambiguous spoken utterance into target concepts and pictograms, as recognized in light of the disclosure of Potter. (Potter, ¶ [0029]).
Regarding claim 20, the rejection of claim 14 is incorporated. Claim 20 is substantially the same as claim 13 and is therefore rejected under the same rationale as above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Kobayashi (U.S. Pat. App. Pub. No. 20170103756) discloses a vehicle mounted information processing device incorporating selectable pictograms.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Sean E. Serraguard whose telephone number is (313)446-6627. The examiner can normally be reached 07:00-17:00 M-F.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel C. Washburn can be reached at (571) 272-5551. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Sean E Serraguard/Primary Examiner, Art Unit 2657
Prosecution Insights