DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claims 6-8 and 11-17 objected to because of the following informalities: The claims are labeled “Currently amended” but no amendment appears in the latest amendment. Appropriate correction is required.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-8, 11-17, and 21 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Representative claim 1 recites “wherein the VR headset is further configured to present sign language to the hearing impaired customer using cartoon-like characters or realistic characters.” It is not clear what the definition is of a “cartoon-like” or “realistic” character is or how to determine whether a character would is cartoon-like or realistic. For purposes of examination, any animated, human-like character will be interpreted to be both cartoon-like and realistic since animated figures take on aspects of cartoons and realism simultaneously.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1-8, 11-17, and 21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Rohatgi et al. (US 2024/0038225) in view of Riahi (US 2014/0270109) in view of Yao (2020/0075011) further in view of Joseph et al. (US 2005/0058264) further in view of Taylor et al. (US 2015/0312468 ).
1. A system for improved service to a hearing impaired customer that connects the hearing impaired customer with an artificial intelligence (“AI”) virtual assistant to address an inquiry, the system comprising:
an artificial intelligence (“AI”) virtual assistant, (A bot is a computer program that operates as an agent for a user or other program or to simulate a human activity. see par. [0003 of Rohatgi et al.])
a virtual reality (“VR”) headset, said VR headset; (User 105 interacts with interactive device 110, which has VA capabilities. Interactive device 110 can be, for example, a bot, a physical robot or a device with a screen, a projector, an augmented reality headset or a virtual reality headset displaying a virtual avatar. see par. [0049] of Rohatgi et al.)
is located at a brick-and-mortar location of an organization; (For example, in a hypothetical situation, if a customer's membership is expiring, a humanoid bot could advise the customer to renew the membership and additionally point the customer to a location in a store where memberships can be renewed. see par. [0091] of Rohatgi et al. Note that Rohatgi teaches an augmented reality VR system so the bot would be operating at the store location)
is worn by the hearing impaired customer to provide a way for the hearing impaired customer to communicate with the AI virtual assistant; and (The gesture for a response to the situation is a gesture that VA designer 195 is suggesting that the bot perform in response to the situation. Examples include body motions (e.g., movement of hands, head, fingers, eyes, legs, or torso), facial expressions ( e.g., a smile, or a blinking of eyes), and other mechanical actions ( e.g., rotation of wheels, or engagement of other mechanical devices). The gestures may include sign language, e.g., American sign language (ASL), for a case where user 105 is hearing-impaired. see par. [0059] of Rohatgi et al.)
comprises one or more cameras that detect hand movements of the hearing impaired customer, said hand movements comprising sign language (Various sensors 137 are utilized for detecting conditions concerning user 105, and sensory parameters (e.g., location, vision, touch, and tap), and may include a microphone, a camera, an accelerometer, a biometric sensor, and detectors of environmental conditions, such as smoke, gas, water and temperature. see par. [0032] of Rohatgi et al.)
an application programming interface (“API”), said API integrates the AI virtual assistant with the VR headset
the API is configured to:
provide two-way communication of data between the VR headset and the AI virtual assistant
transfer an electronic record or the hand movements of the hearing impaired customer, detected by the VR headset, to the AI virtual assistant
the AI virtual assistant comprises an AI algorithms and/or machine learning (“ML”) and is configured to:
identify the hand movements of the hearing impaired customer as found in the electronic record that is transferred (The gesture for a response to the situation is a gesture that VA designer 195 is suggesting that the bot perform in response to the situation. Examples include body motions (e.g., movement of hands, head, fingers, eyes, legs, or torso), facial expressions ( e.g., a smile, or a blinking of eyes), and other mechanical actions ( e.g., rotation of wheels, or engagement of other mechanical devices). The gestures may include sign language, e.g., American sign language (ASL), for a case where user 105 is hearing-impaired. see par. [0059] of Rohatgi et al.)
Rohatgi teaches a VR headset augmented reality system in which a hearing impaired user is able to interact with a bot using sign language. Rohatgi teaches an artificial intelligence bot that communicates with sign language, but not necessarily translating a live conversation with another person as taught by Yao
translate a request for assistance from sign language into text, said request is made by the customer that is using sign language, and said customer is wearing the VR (When the user terminal 1 captures a video of a scenario in front of the user in real time, the user views a sign language video in which a AR sign language animation is displayed in the video through the user terminal 1. The following various sign language information processing methods of the present disclosure may be completed only in the user terminal 1, or may be jointly performed by the user terminal 1 and the server 2. see par. [0094] of Yao) headset at a brick-and-mortar location of an organization; (The system of Yao functions over wireless communications and can operate anywhere communication signals are available including brick and mortar locations The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. see par. [0175] of Yao)
and the AI virtual assistant and the VR headset, integrated with each other using the API, (In an implementation, the process of sign language information processing may mainly rely on the user terminal 1 for processing. see par. [0095] of Yao) are configured to conduct real-time communication between the agent and the customer to resolve the request. (When the user terminal 1 captures a video of a scenario in front of the user in real time, the user views a sign language video in which a AR sign language animation is displayed in the video through the user terminal 1. The following various sign language information processing methods of the present disclosure may be completed only in the user terminal 1, or may be jointly performed by the user terminal 1 and the server 2. see par. [0094] of Yao)
wherein the VR headset is further configured to present sign language to the hearing impaired customer using cartoon-like characters or realistic characters (When the user terminal 1 captures a video of a scenario in front of the user in real time, the user views a sign language video in which a AR sign language animation is displayed in the video through the user terminal 1. The following various sign language information processing methods of the present disclosure may be completed only in the user terminal 1, or may be jointly performed by the user terminal 1 and the server 2. see par. [0094] of Yao)
It would have been obvious to one of ordinary skill in the art at the time of filing to expand the system of Rohatgi et al. to include an on command virtual sign language interpretation system as taught by Yao. One of ordinary skill in the art at the time of filing would have been motivated to expand the system of Rohatgi in this way in this way since the system of Rohatgi is capable of capturing gestures and offering live translation services would make the augmented system more versatile.
Rohatgi et al. in view of Yao does not explicitly teach a customer service system as taught by Riahi below
determine a team in the organization that can assist the hearing impaired customer; (The results of the IVR transaction may be sent back to the CIM module. For instance, the customer may have a question that cannot be addressed by the self-service IVR module, in which case the CIM module may route the customer's call to a live agent. The live agent may then get a prompt from the CIM module about the incoming call (for example, saying that the customer has just used the self-service IVR module and has a question for a live agent). see par. [0171] of Riahi)
transfer the hearing impaired customer to an agent in the team said agent does not know sign language (The automated customer greeting module 1110 may serve as a first point of contact between a customer and the contact center. The customer greeting module 1110 may serve several functions, such as identifying the customer, identifying the reason for the interaction, and choosing a first routing of the interaction to a more appropriate component, contact, live agent, etc., depending on factors such as the nature of the interaction. In one embodiment, the automated customer greeting module 1110 may be the iCFD (intelligent Customer Front Door)...iCFD is a call routing application that collects a customer's intent (based on what they say) using speech recognition, determines how to treat the call based on that input and feedback from the back-end systems, customer relations management (CRM) systems, and other data, and determines proactively where the customer should be routed and what service should be provided, whether live agent assisted, self service, or (with appropriate extensions) to an automated agent. see par. [0167-0168] of Riahi)
provide the first text to the agent
conduct a real-time two-way communication between the agent and the hearing impaired customer that concludes with a resolution of the inquiry
determine a team in the organization that can assist the customer; and
transfer the customer to an agent in the team (The results of the IVR transaction may be sent back to the CIM module. For instance, the customer may have a question that cannot be addressed by the self-service IVR module, in which case the CIM module may route the customer's call to a live agent. The live agent may then get a prompt from the CIM module about the incoming call (for example, saying that the customer has just used the self-service IVR module and has a question for a live agent). see par. [0171] of Riahi)
One of ordinary skill in the art at the time of filing would have been motivated to expand the system of Rohatgi et al. in view of Yao in this way since Rohatgi is capable of communicating over telephone networks (The processing component 802 typically controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communications, camera operations, and recording operations. see par. [0169] of Rohatgi et al.) in order to expand the capabilities of the virtual agent of Rohatgi that is providing in store services (see Fig. 5 of Rohatgi et al.)
Rohatgi et al. in view of Yao further in view of Riahi does not explicitly teach confirming with a customer that their message was understood.
confirm with the hearing impaired customer that the AI virtual assistant correctly understands the inquiry
Joseph et al. teaches confirming a customer’s input in an automated system. (see Fig. 3B of Joseph et al.). One of ordinary skill in the art at the time of filing would have been motivated to expand the system of Rohatgi et al. in view of Yao further in view of Rao since Riahi teaches proactively determining a customer’s intent and confirming would be an obvious way to guarantee that the system operates correctly (iCFD is a call routing application that collects a customer's intent (based on what they say) using speech recognition, determines how to treat the call based on that input and feedback from the back-end systems, customer relations management (CRM) systems, and other data, and determines proactively where the customer should be routed and what service should be provided, whether live agent assisted, self service, or (with appropriate extensions) to an automated agent.. see par. [0168] of Riahi).
Rohatgi et al. in view of Yao further in view of Riahi further in view of Josephe et al. do not explicitly disclose API keys which is taught by Taylor et al.
Wherein the API is secured with API keys, said API keys configured to verify the VR headset and the AI virtual assistant (For example, a server or authentication server can authenticate a user via security certificates, HTTPS, or API keys. API keys can include various encryption standards Such as, e.g., Advanced Encryption Standard (AES). Data resources can be sent over Transport Layer Security (TLS) or Secure Sockets Layer (SSL). The client 102 and server 106 can be deployed as and/or executed on any type and form of computing device see par. [0031]-[0032] of Taylor et al.).
It would have been obvious to one of ordinary skill in the art at the time of filing to include API keys as taught by Taylor et al. since the claimed invention is merely the combination of old elements and in the combination each element merely would have performed the same function as it did separately, and one of ordinary skill in the art would have recognized that the results of the combination were predictable.
Note that some limitations are interpreted as field of use limitations that are not functionally limiting on the claims. In particular, the system functions the same whether or not a customer is “hearing-impaired.” What the system adapts to is the usage of sign language to communicate. Therefore, the level if hearing impairment does not currently functionally limit the instant claims. Similarly, the language ability of the agent does not functionally limit the claims since the functions are merely to translate sign language into text which, as currently claimed, would happen regardless of the agent’s knowledge of sign language.
2. The system of claim 1 wherein the sign language is American Sign Language (“ASL”). (The examiner takes official notice that American Sign Language existed at the time of invention and it would have been obvious to one of ordinary skill in the art to use American Sign Language in the invention filed with the United States of America Patent Office in order to make the system usable with Americans.)
3. The system of claim 1 wherein the agent is in a financial center, a store, a contact center, or a call center. (For example, a contact center for a bank may tell callers, via the IVR script, to “press 1 if they wish to get an account balance. see par. [0074] of Riahi)
4. The system of claim 1 wherein the brick-and-mortar location is a financial center, and the AI virtual assistant is a virtual financial assistant (“VFA”). (For example, a contact center for a bank may tell callers, via the IVR script, to “press 1 if they wish to get an account balance. see par. [0074] of Riahi)
Note that terminology used is interpreted in view of the disclosure and the described functionality. Therefore a “virtual financial assistant” is being interpreted as a software agent capable of functions specifically recited in the instant claim.
5. The system of claim 1 wherein the VR headset comprises VR, augmented reality (“AR”), extended reality (“XR”), spatial computing, or a combination herein. (superimposing and displaying an augmented reality AR sign language animation corresponding to the voice information on a gesture area corresponding to the speaking object to obtain a sign language video see par. [0010] of Yao)
6. The system of claim 1 wherein the system for improved service to the hearing impaired customer leads to an improved level of customer satisfaction. (This limitation is considered a field of use limitation by the Examiner. Language that suggests or makes optional but does not require steps to be performed or does not limit a claim to a particular structure does not limit the scope of a claim or claim limitation. Please see MPEP 2111.02 II, 2103 (C), and 2114.)
7. The system of claim 1 wherein the real-time communication between the agent and the hearing impaired customer comprises: sign language, from the hearing impaired customer, that is translated by the VR headset worn by the hearing impaired customer into text that is transmitted using the AI virtual assistant to the agent as text or as speech (It can be understood that the gesture action information can be understood as a gesture action image. Recognizing the gesture action information of the user himself in the video information may be to recognize an image of a preset position. For example, gesture action information obtained in a lower edge area of a video frame is taken as the gesture action information of the user himself. see par. [0117] of Yao) ; and typed text, from the agent (a speech synthesis module 1170 for converting text to speech for delivery to the customer, see par. [0166] of Yao), that is transmitted by the AI virtual assistant to the VR headset, worn by the hearing impaired customer, as sign language or text. (superimposing and displaying an augmented reality AR sign language animation corresponding to the voice information on a gesture area corresponding to the speaking object to obtain a sign language video see par. [0010] of Yao)
8. The system of claim 1 wherein the real-time communication between the agent and the hearing impaired customer comprises: sign language, from the hearing impaired customer, that is translated by the VR headset worn by the hearing impaired customer into text that is transmitted using the AI virtual assistant to the agent as text or as speech, said VR headset being a first VR headset (In an embodiment, user sound attribute information preset by the user may be obtained, and then the user voice information is played with the user sound attribute. The user sound attribute information may be understood as a characteristic configuration of a sound that the user wants to output, which is configured in advance, for example, an old person's voice, a youth's voice, a child's voice, a man's voice, or a woman's voice, thereby realizing a customization of the user's voice. see par. [0123] of Yao); and speech, spoken by the agent, that is translated by a second VR headset worn by the agent into text and transmitted to the hearing impaired customer on the VR headset as sign language or text. (The customer portal module 110 may provide, for example, an avatar for voice and/or Video communication (such as over the web or a Smartphone) as implemented by an avatar module 210.see par. [00095] of Riahi)
Claims 21 and 11-17 recite substantially similar limitations to claims 1-9 and are rejected for the same reasoning.
Response to Arguments
Applicant’s arguments with respect to the claims have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to MINNAH L SEOH whose telephone number is (571)270-7778. The examiner can normally be reached Monday - Friday 9-4.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Director Joseph Thomas can be reached at (571) 272-8004. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/MINNAH L SEOH/Supervisory Patent Examiner, Art Unit 3618
Prosecution Insights