Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 4, 6-9, and 11 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Rahmani et al. (U.S. Patent Application Pub. No. 2023/0085161, hereinafter “Rahmani”).
In regard to claim 1, Rahmani discloses a communication assist device for communication using a sign language (Fig. 7, 700), the communication assist device comprising:
a sign language recognition module for extracting a sign language sentence from an analyzed motion of a user in image data (signs in a video are converted to text, paragraphs [0051] and [0069]); and
a display for displaying the extracted sign language sentence (the text is displayed on a display, paragraph [0071]).
In regard to claim 2, Rahmani discloses an STT module for converting voice data into text data (an ASR engine converts audio data into text, paragraph [0046]); and
a sign language generation module for converting voice data into sign language data (sign/text engine circuitry generates sign language data based on the meaning of text recognized from the speech and a volume, cadence, tone, and/or emotion of the speech, paragraph [0047]).
In regard to claim 4, Rahmani discloses a text input module for providing a user interface for the user to input a text to the display, wherein the text input module is activated when the sign language recognition module fails to extract the sign language sentence (if the translation is incorrect, the user can enter a correction into their device, paragraph [0074]; via keyboard, paragraph [0101]).
In regard to claim 6, Rahmani discloses the sign language recognition module is configured to:
partition the image data into a plurality of data segments (a video is segmented with time stampings, paragraph [0057]);
determine a recognition accuracy of each gloss of the plurality of segments (confidence scores are generated for gloss labels for each of the segments, paragraph [0057]); and
extract a sign language sentence based on a gloss with the recognition accuracy greater than a predetermined value among glosses of the plurality of segments (a continuous full sentence is determined based on the highest confidence scores, paragraphs [0058-0059]).
In regard to claim 7, Rahmani discloses the recognition accuracy is determined based on a similarity between a gloss of a segment and a similar gloss, and the similar gloss is a gloss that is most similar to the gloss of the segment (the confidence score is based on a similarity between the determined gloss and the correct gloss label, paragraphs [0055-0056]).
In regard to claim 8, Rahmani discloses the sign language recognition module extracts skeleton information for tracking a motion of the user by detecting joint part of the user from the image data and compares a gloss of the user according to the skeleton information with a similar gloss (joints from a human skeleton are used as key points to determine the most similar gloss, paragraphs [0048], [0052], and 0055]).
In regard to claim 9, Rahmani discloses the display displays a message for requesting retransmission of a sign language sentence, when every recognition accuracy of the glosses of the plurality of segments is smaller than a predetermined value (if the translation cannot be completed, an error message is produced and transmitted to request correction, paragraph [0074]).
In regard to claim 11, Rahmani discloses when the glosses of the plurality of segments include a first gloss with a recognition accuracy greater than the predetermined value and a second gloss with a recognition accuracy smaller than the predetermined value, the sign language recognition module determines a plurality of gloss candidates replacing the second gloss based on the first gloss and extracts a sign language sentence based on a gloss candidate selected from the plurality of gloss candidates and the first gloss (see Fig. 4, using a Non-Maximum Suppression algorithm, gloss candidates with a high confidence score are selected to replace overlapping gloss candidates with a low confidence score, paragraphs [0057-0058]).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3, 10 and 12-13 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rahmani, in view of Vieira Rocha et al. (U.S. Patent Application Pub. No. 2022/0188538, hereinafter “Vieira Rocha”).
In regard to claim 3, Rahmani does not disclose a word card selection module for providing a word card selectable for the user to the display.
Vieira Rocha discloses a communication assist device for communication using a sign language, comprising:
a word card selection module for providing a word card selectable for the user to the display, wherein the sign language recognition module extracts the sign language sentence based on the selected word card (see Fig. 6, when inputting a sentence 606 using sign language, a button 610 causes a pop-up display element overlaid on GUI 600 comprising selectable alternative words to determine the sentence, paragraphs [0078-0080]; the pop-up display element overlaid on GUI 600 is considered equivalent to the claimed “word card”).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to provide a word card selectable for the user to the display, because they provide an improved user interface that increases functionality, accuracy, and ease of use, as taught by Vieira Rocha (paragraph [0087]).
In regard to claim 10, Rahmani discloses the sign language recognition module extracts a sign language sentence based on a gloss with the recognition accuracy greater than a predetermined value (a continuous full sentence is determined based on the highest confidence scores, paragraphs [0058-0059]).
Rahmani does not disclose the sign language recognition module extracts a sign language sentence based on a previous conversation content.
Vieira Rocha discloses a communication assist device for communication using a sign language comprising a sign language recognition module that extracts a sign language sentence based on a previous conversation content (a model uses context provided by previous words to predict an output sentence, paragraph [0058]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to extract a sign language sentence based on a previous conversation content, because it would allow the correct sentence to be recognized even if portions of the sign language input were not captured, as taught by Vieira Rocha (paragraphs [0025-0027]).
In regard to claim 12, Rahmani discloses when the glosses of the plurality of segments include a first gloss with a recognition accuracy greater than the predetermined value and a second gloss with a recognition accuracy smaller than the predetermined value, the sign language recognition module determines a plurality of gloss candidates replacing the second gloss based on the first gloss and extracts a sign language sentence based on a gloss candidate selected from the plurality of gloss candidates and the first gloss (see Fig. 4, using a Non-Maximum Suppression algorithm, gloss candidates with a high confidence score are selected to replace overlapping gloss candidates with a low confidence score, paragraphs [0057-0058]).
Rahmani does not disclose the sign language recognition module selects a gloss candidate based on a previous conversation content.
Vieira Rocha discloses a communication assist device for communication using a sign language comprising a sign language recognition module that selects a gloss candidate based on a previous conversation content (a model uses context provided by previous words to predict an output sentence, paragraph [0058]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to select a gloss candidate based on a previous conversation content, because it would allow the correct sentence to be recognized even if portions of the sign language input were not captured, as taught by Vieira Rocha (paragraphs [0025-0027]).
In regard to claim 13, Rahmani does not disclose the sign language recognition module determines a priority order for the plurality of glass candidates according to a similarity to the second gloss, and
wherein the display displays the plurality of gloss candidates according to a priority order.
Vieira Rocha discloses the sign language recognition module determines a priority order for the plurality of glass candidates according to a similarity to the second gloss (candidates are ranked by their likelihood, paragraph [0062]), and
wherein the display displays the plurality of gloss candidates according to a priority order (the most likely candidates are displayed for selection by the user, paragraphs [0078-0080]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to display the plurality of gloss candidates according to a priority order, because they provide an improved user interface that increases functionality, accuracy, and ease of use, as taught by Vieira Rocha (paragraph [0087]).
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rahmani, in view of Dharmarajan Mary (U.S. Patent Application Pub. No. 2017/0277684).
In regard to claim 5, Rahmani discloses a communication module for controlling the communication assist device to be communicatively connected with external device (communication server 302, paragraph [0044]).
Rahmani does not disclose the communication module controls the communication assist device to be connected with the external device when the sign language recognition module fails to extract the sign language sentence.
Dharmarajan Mary discloses a communication assist device comprising a communication module wherein the communication module controls the communication assist device to be connected with the external device when the sign language recognition module fails to extract the sign language sentence (when an input device receives an indication that a sign language interpretation is incorrect, a communication component transmits information to a communications device, paragraph [0059]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to control the communication assistant device to be connected with an external device when the sign language recognition module failed to extract the sign language sentence, because it would allow the system to store information in order to reduce errors and improve the speed of communication for that user, as taught by Dharmarajan Mary (paragraph [0030]).
Claim(s) 14 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rahmani, in view of Ghosh et al. (U.S. Patent Application Pub. No. 2023/0077446, hereinafter “Ghosh”).
In regard to claim 14, Rahmani does not disclose the display is a transparent display.
Ghosh discloses a sign language conversation device comprising a transparent display (paragraph [0061]).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to utilize a transparent display, because it would allow the display to be incorporated in smart glasses, which provides advantages such as portability, ease of use, etc. as taught by Ghosh (paragraph [0014]).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Jang et al., Robinson, Peralta et al., Kelly, Matula et al., Lin et al., Menefee et al., and Green, Jr. et al. disclose additional sign language communication assist devices.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to BRIAN LOUIS ALBERTALLI whose telephone number is (571)272-7616. The examiner can normally be reached M-F 8AM-3PM, 4PM-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Bhavesh Mehta can be reached at 571-272-7453. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
BLA 7/25/26
/BRIAN L ALBERTALLI/Primary Examiner, Art Unit 2656
Prosecution Insights