DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This Non-Final Office Action is in response to the RCE filed on11/17/2025. Claims 1-20 have been examined and are pending. Claims 1, 10, and 18 are amended.
Priority
Application 18/171,038 filed 02/17/2023 claims priority to provisional application 63/268,160 filed 02/17/2022.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 11/17/2025 has been entered.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to judicial exception (i.e., a law of nature, a natural phenomenon, or an abstract idea) without significantly more. Claims 1-20 are directed to a system, method, or product which are/is one of the statutory categories of invention. (Step 1: YES).
Claims 1, 10 and 18 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim recites a method and computing device for facilitating customer-agent interactions. For Claims 1, 10 and 18 the limitations of (Claim 1 being representative):
facilitating, […], an interaction between a user and an agent upon receiving a request for initiating an interaction from the user;
receiving[…], a […] workflow comprising a set of instructions from the agent, wherein the agent selects the […] workflow from a plurality of […] workflows based, at least in part, on interpreting a user objective for initiating the interaction;
receiving, […], a viewfinder frame […] associated with the user subsequent to initializing an […] session by the user in response to executing a first instruction from the set of instructions;
wherein viewfinder frames are neither stored nor forwarded to the agent to ensure that the user’s privacy is not compromised; and
iteratively performing, […], a plurality of operations until each instruction from the set of instructions is executed, the plurality of operations comprising:
[…] analyzing, […], the viewfinder frame to determine a subsequent instruction to be executed by the user from the set of instructions,
facilitating, […], a display of an […] image frame […], wherein the […] image frame is generated based, at least in part, on the subsequent instruction,
determining, […], an execution status of the subsequent instruction by monitoring the user while the user executes the subsequent instruction, the execution status indicating whether the subsequent instruction is one of successful and unsuccessful, and
transmitting[…], a notification indicating the execution status to the agent, as drafted, are processes that, under the broadest reasonable interpretation, covers certain methods of organizing human activity (i.e., commercial or legal interactions and/or managing personal behavior including following rules or instructions) but for recitation of generic computer components. The Examiner notes that “certain method[s] of organizing human activity” includes a person's interaction with a computer (see MPEP 2106.04(a)(2)(II)). That is, other than reciting a system implemented by an Augmented Reality (AR)- based workflow, electronic device, AR session, AR image, processor, memory, and non-transitory computer-readable storage medium, the claimed invention amounts to managing personal behavior or interaction between people and/or commercial or legal interactions. For example, but for the Augmented Reality (AR)- based workflow, electronic device, AR session, AR image, processor, memory, and non-transitory computer-readable storage medium, this claim encompasses a person to facilitate an interaction between a user and an agent upon receiving a request for initiating an interaction from the user, receiving a workflow comprising a set of instructions from the agent wherein the agent selects the workflow from a plurality of workflows based on interpreting a user objective for initiating the interaction, receive a frame associated with the user in response to executing an where the frames are neither stored nor forwarded, iteratively performing operations until each instruction is executed comprising: analyzing the frame to determine a subsequent instruction to be executed, displaying an image frame generated on the subsequent instruction, determining a status of the subsequent instruction by monitoring the user, the status indicating whether the subsequent instruction is one of successful and unsuccessful, and transmitting a notification indicating the execution status based on this data in the manner described in the identified abstract idea, supra. If a claim limitation, under its broadest reasonable interpretation, covers commercial or legal interactions and/or managing personal behavior or interactions between people but for the recitation of generic computer components, then it falls within the “certain methods of organizing human activity” grouping of abstract ideas. Accordingly, Claims 1, 10 and 18 recite an abstract idea. (Step 2A- Prong 1: YES. The claims recite an abstract idea).
This judicial exception is not integrated into a practical application. Claims 1, 10 and 18 recites the additional elements of a processing system (Claims 1), Augmented Reality (AR)- based workflow (Claims 1, 10 and 18), electronic device (Claims 1, 10 and 18), AR session (Claims 1, 10 and 18), AR image (Claims 1, 10 and 18), a processor (Claim 10, and 20), memory (Claims 10), a non-transitory computer-readable storage medium (Claim 18) that implements the identified abstract idea. These additional elements are not described by the applicant and are recited at a high-level of generality (i.e., one or more generic computers performing a generic computer functions) such that it amounts no more than mere instructions to apply the exception using a generic computer components. Accordingly, even in combination these additional elements do not integrate the abstract idea into a practical application because it does not impose any meaningful limits on practicing the abstract idea. Claims 1, 10 and 18 are directed to an abstract idea. (Step 2A-Prong 2: NO: the additional claimed elements are not integrated into a practical application).
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception. As discussed above with respect to integration of the abstract idea into a practical application, the additional elements of a processing system (Claims 1), Augmented Reality (AR)- based workflow (Claims 1, 10 and 18), electronic device (Claims 1, 10 and 18), AR session (Claims 1, 10 and 18), AR image (Claims 1, 10 and 18), a processor (Claim 10, and 20), memory (Claims 10), a non-transitory computer-readable storage medium (Claim 18), to perform the noted steps amounts to no more than mere instructions to apply the exception using a generic computer component (See Spec. Paragraphs [0027, 0031, 0034-0035]. Mere instructions to apply an exception using a generic computer component cannot provide an inventive concept (“significantly more’). Accordingly, even in combination, these additional elements do not provide significantly more. As such claims 1, 10 and 18 are not patent eligible. (Step 2B: NO. The claims do not provide significantly more).
Dependent Claims 2-9, 11-17 and 19-20 are similarly rejected because they either further define/narrow the abstract idea of independent claims 1, 10 and 18 as discussed above. Claim(s) 2 & 11 merely describe(s) an option to initialize the session. Claim(s) 6 & 15 merely describe(s) determining promotional content. Claims(s) 7 & 16 merely describe(s) overlaying instruction on the frame. Claim(s) 8, 17 & 20 merely describe(s) determining that a status is unsuccessful and displaying additional instructions. what the first set of information describing the potential candidate, second notification, and match with a request that the users recognize each other include. Therefore claims 2, 6-8, 11, 15-17, and 20 are considered patent ineligible for the reasons given above.
Dependent Claim(s) 3-5, 9, 12-14, and 19 recite limitations that further define the abstract idea noted in independent claims 1, 10, and 18. In addition, it recites the additional elements of a database, and virtual agent. The database, and virtual agent are recited at a high level of generality such that it amounts to no more than mere instructions to apply the exception using a generic computing component. Even in combination, these additional elements do not integrate the abstract idea into a practical application and do not amount to significantly more than the abstract idea itself. Therefore, dependent claims 2-9, 11-17 and 19-20 are considered patent ineligible for the reasons given above.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5, 8, 9, 10, 14, 17, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Amir (US 20210174371 A1), in view of Avila (US 20180278750 A1), and in further view of Chachek (US 20200302510 A1) .
Regarding Claim 1,
Amir discloses, A computer-implemented method, the comprising: facilitating, by a processing system, an interaction between a user and an agent upon receiving a request for initiating an interaction from the user; " Support session 20 is initiated when a user 33 calls or otherwise contacts TSC 36 using a mobile device 31. Initiation (step A1) may be performed over a cellular and/or landline network, or other communication channels e.g., satellite communication, voice over IP, etc. When a call is received, TSC 36 sends a message (step A2) to the mobile device 31. The message may be SMS, email, WhatsApp, etc. and comprises a link (e.g. URL) for commencing a the support session. Upon opening the link (step A3), mobile device 31 accesses a remote server 36s over a data network 32, wherefrom video support session 21 setup instructions/code are sent to the mobile device 31 to establish the session (step A4). The remote server 36s may be implemented as part of the support center and/or in a cloud computing infrastructure accessible for both the users and the support center" (Amir Par. 0052).
receiving, by the processing system, an augmented reality (AR)-based workflow comprising a set of instructions from the agent, wherein the agent selects the AR-based workflow from a plurality of AR-based workflows based, at least in part, on interpreting a user objective for initiating the interaction; "Additionally, or alternatively, speech analysis tools may be used to analyze the user's speech to identify keywords within the speech and aid the computer vision tool as it processes image data 33i for relevant objects/elements within the image sensor field of view. For example, if the speech recognition tool identifies words such as internet/network and communication/connectivity in the user' speech, it may guide the computer vision tool to look for LAN sockets or cables, Wi-Fi antennas and/or LEDs indications. Optionally, the keywords may be typed by agent 36p. Upon identifying the relevant objects in the image data 33i, the TSC system using the computer vision tool can analyze the object's setup/configuration and automatically identify potential issues/defects therein. Display device 36d may be used to present to the identified object to the agent. Once the issue is identified, agent 36p may instruct the user on how to solve it. If the solution is relatively simple, (e.g., press the power switch), agent 36p may provide verbal instructions. If user 33 is unable to carry out the verbal instructions, or the instructions are relatively complex, agent 36p may generate an instructive augmented reality video stream using one or more markers 39 and trackers (step S8). The markers are superimposed onto the image data and displayed on the user's mobile device in real time to provide additional guidance. The agent may alternatively superimpose annotations as discussed and described above with respect to FIGS. 1A-1C. Optionally, TSC 36 may query database 36r for a best working solution (step S7), based on the object's determined issues/defects, and transmits the best working solution to user 33. The instructions may comprise textual, auditory and/or annotated/augmented content. Agent 36p may provide user 33 with some (including one) or all types of instructive content, and/or limit the content to include some or the entire set of instructions." (Amir Par. 0066-0068).
receiving, by the processing system, a viewfinder frame from an electronic device associated with the user subsequent to initializing an AR session by the user in response to executing a first instruction from the set of instructions; and "Upon establishing a video support session, the support center processes and analyzes the sounds and image data received from the remote end user. The support center provides tools for conveying instructions to the remote user. For example, where image data is comprised of one or more still images, the support center may add annotations, e.g., text, signs and/or symbols to the image data. Where image data is comprised of real-time video stream or video frames, the support center may superimpose a movable augmented indicator onto the image data. The annotated or superimposed image data is presented on the display of the user's mobile device. When the remote user successfully resolves the problem by following the annotated/superimposed instructions, the problem and solution is stored in a cloud or other database record system. By storing various problems and solutions, a database of working solutions is gradually established. The database may be used by the support center to more quickly and efficiently solve future problems. Alternatively, the database may form an artificial intelligence, whereby the artificial intelligence and not an agent of the support center solves the problem using the image data and relays the instructions to the user using annotations or a moveable augmented indicator as described above" (Amir Par. 0046).
iteratively performing, by the processing system, a plurality of operations until “FIG. 5B illustrates a decision tree used by support system 50 for determining the sequential instructions required to remedy the inoperability of an appliance. The input for the decision tree may be data derived from image data captured by image sensor 31c. For example, the image data may depict a current state of the inoperative product. The nodes in FIG. 5B represent operative states of the inoperative product and the lines represent the actions or the steps the user needs to complete to render the inoperative product operative.” (Amir Par. 0091). “If it is determined in step 50 that the best past solution obtained in steps 48-49 resolved the user's problem, a new database record 51 is constructed in step 54, and then stored in the database of the system for use in future trouble shooting sessions. The new database record may include one or more annotated images, a video showing how to fix the problem (with or without AR markers), text and/or audible instructions. If the best past solution is unsuccessful, other high-ranking solutions are obtained from the database, and presented in attempt to resolve the problem. Steps 48 to 50 may be repeated for each solution until a successful solution is found. Alternatively, or concurrently, speech analysis 12s, image recognition 12i, and steps 41 to 46 can be carried out to determine alternative problems/defects in the object 33e. (Amir Par. 0083).
electronically analyzing, by the processing system, the viewfinder frame to determine a subsequent instruction to be executed by the user from the set of instructions, “FIGS. 8A-8D illustrate another application of image processing system 60, wherein the image processing system 60 employs artificial intelligence during technical support. The image processing system comprises at least one processor configured to: receive image data captured by an image sensor of a mobile device, the image data including images of an inoperative appliance in an environment of a user; perform image recognition on the image data to determine an identity of the inoperative appliance and a likely cause of inoperability; retrieve a plurality of sequential instructions to be provided for enabling a user to complete a plurality of sequential actions in order to remedy the inoperability; cause the mobile device to sequentially display the plurality of sequential instructions; detect that the inoperative appliance is outside a field of view of the image sensor, based on the image data and during execution of the sequential actions; suspend display of additional sequential instructions while the inoperative appliance is outside of the field of view; detect when the inoperative appliance returns to the field of view after suspending display; and resume display of the additional sequential instructions after the inoperative appliance is detected to return to the field of view.” (Amir Par. 0122).
facilitating, by the processing system, a display of an AR image frame on the electronic device, wherein the AR image frame is generated based, at least in part, on the subsequent instruction, “As seen in FIG. 8A, once an appliance 83e and relevant functional elements 83f, 83h, and 83i are detected and identified in the live video stream, and a likely source of inoperability is assessed, a plurality of sequential instructions 85 for repairing/assembling the object are displayed on the mobile device. A likely source of error may be derived from database 36r as discussed above, or deduced from the operational state of the elements. For example, if the processor detects, using the data processing techniques described above, that the appliance is not connected to a power source, it may deduce that a lack of power is the cause of inoperability. Instructions 85 may be displayed one at a time, or multiple instructions may be displayed simultaneously. FIG. 8A illustrates yet another embodiment wherein at least two instructions and a moveable augmented indicator 84a are simultaneously displayed on a mobile device 81.” (Amir Par. 0123).
determining, by the processing system, an execution status of the subsequent instruction by monitoring the user while the user executes the subsequent instruction, the execution status indicating whether the subsequent instruction is one of successful and unsuccessful, and “Once retrieved, the processor annotates the image with one or more instructions for correcting the source of inoperability. FIG. 13B illustrates an annotated image wherein functional elements 103k, 103l have been identified as being in the wrong port. Annotation 134 therefore directs the user to switch the position of elements 103k and 103l to correct the configuration. Once corrected, the user may submit another image of the configuration to the processor, and the processor may notify the user that the step was completed successfully using positive feedback, e.g. check mark 135 (FIG. 13C).” (Amir Par. 0147). “The new database record may include one or more annotated images, a video showing how to fix the problem (with or without AR markers), text and/or audible instructions. If the best past solution is unsuccessful, other high-ranking solutions are obtained from the database, and presented in attempt to resolve the problem. Steps 48 to 50 may be repeated for each solution until a successful solution is found” (Amir Par. 0083).
Amir discloses facilitating an interaction between a user and an agent upon receiving a request for initiating an interaction from the user, receiving an augmented reality based workflow comprising a set of instructions from the agent, wherein the agent selects the AR-based workflow from a plurality of AR-based workflows based on interpreting a user objective for initiating the interaction, receiving a viewfinder frame from an electronic device associated with the user subsequent to initializing an AR session by the user in response to executing a first instruction from the set of instructions, iteratively performing a plurality of operations until each instruction from the set of instructions is executed, the plurality of operations comprising electronically analyzing the viewfinder frame to determine a subsequent instruction to be executed by the user from the set of instructions, facilitating a display of an AR image frame on the electronic device, wherein the AR image frame is generated based, at least in part, on the subsequent instruction, and determining an execution status of the subsequent instruction by monitoring the user while the user executes the subsequent instruction, the execution status indicating whether the subsequent instruction is one of successful and unsuccessful. Amir fails to disclose transmitting a notification indicating the execution status to the agent, and wherein viewfinder frames are neither stored nor forwarded to the agent to ensure that the user's privacy is not compromised. Avila, however, does disclose:
transmitting, by the processing system, a notification indicating the execution status to the agent. “The routing determination component 114 is configured to output a routing determination in accordance with a state of the help desk case. For instance, in an example where a help desk request is being created, an interaction between the matched support agent and the customer may be initiated through a modality of the help desk service. In an example, where a first support agent is involved in a communication with a customer, a second support agent (e.g. matched support agent) may be added/patched into the communication. In examples where a routing determination relates to generation of a follow-up inquiry for an unresolved help desk case, the routing determination component 114 may automatically provide a notification to a support agent to follow-up with a customer. For instance, an agent may be automatically assigned to follow-up with a customer for an unresolved request. In another instance where an agent is assigned to a help desk case, an agent may automatically receive a reminder to follow-up with a customer. In further examples, a follow-up inquiry may automatically be transmitted based to a customer and/or support agent. For instance, the routing determination model may identify that a follow-up is needed for a help desk case, evaluate presence information for the customer and/or support agent and transmit a communication. This processing may be useful in keeping resolution of the help desk case as a priority” (Avila Par. 0038).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir with transmitting a notification indicating the execution status to the Agent of Avila to identify that a follow-up is needed for a help desk case, and keep resolution of a case as a priority (Avila Par. 0038).
The combination of Amir and Avila fail to disclose wherein viewfinder frames are neither stored nor forwarded to the agent to ensure that the user's privacy is not compromised. Chachek, however, does disclose augmented reality navigating a location. Chachek discloses,
wherein viewfinder frames are neither stored nor forwarded to the agent to ensure that the user's privacy is not compromised. “In some embodiments, some or most of all of the computer vision analysis of captured image footage or video footage, may be performed locally within the end-user device of each such user of the crowd-sourced participants; thereby reducing the need and the cost to send and receive large amounts of image data or video data, or to store or process them at cloud-based servers or store-located servers; and/or increasing the level of privacy that is provided to users, as the system need not continuously share or upload a live stream of video or images, which involves a greater exposure to the privacy of the user himself as well as other in-store customers around him, but rather, the system only selectively uploads particular images or image-portions that depict a recognized product, rather than images of faces or people); and/or also increasing the speed of detecting products and localizing items within the store due to parallel computing or distributed computing efforts (e.g., ten end-user devices are performing locally and in parallel in near-real-time their computer vision analysis on their ten respective images or video streams; instead of the latency that would be created by each device having to upload its images or videos to a processing queue at a remote server)” (Chachek Par. 0012).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir and Avila with neither storing nor forwarding data to ensure that the user's privacy is not compromised of Chachek to maintain privacy and/or anonymity of users (Chachek Par. 0177).
Regarding Claim 10, and Claim 18
Amir discloses, An apparatus, comprising: at least one processor; and a memory having stored therein machine executable instructions, that when executed by the at least one processor, cause the apparatus, at least in part, to: (Amir Par. 0005)
A non-transitory computer-readable storage medium comprising computer-executable instructions that, when executed by at least a processor of an apparatus, cause the apparatus to perform a method comprising: (Amir Par. 0006)
facilitate an interaction between a user and an agent upon receiving a request for initiating an interaction from the user; “Support session 20 is initiated when a user 33 calls or otherwise contacts TSC 36 using a mobile device 31. Initiation (step A1) may be performed over a cellular and/or landline network, or other communication channels e.g., satellite communication, voice over IP, etc. When a call is received, TSC 36 sends a message (step A2) to the mobile device 31. The message may be SMS, email, WhatsApp, etc. and comprises a link (e.g. URL) for commencing a the support session. Upon opening the link (step A3), mobile device 31 accesses a remote server 36s over a data network 32, wherefrom video support session 21 setup instructions/code are sent to the mobile device 31 to establish the session (step A4). The remote server 36s may be implemented as part of the support center and/or in a cloud computing infrastructure accessible for both the users and the support center" (Amir Par. 0052).
receive an augmented reality (AR)-based workflow comprising a set of instructions from the agent, wherein the agent selects the AR-based workflow from a plurality of AR-based workflows based, at least in part, on interpreting a user objective for initiating the interaction; "Additionally, or alternatively, speech analysis tools may be used to analyze the user's speech to identify keywords within the speech and aid the computer vision tool as it processes image data 33i for relevant objects/elements within the image sensor field of view. For example, if the speech recognition tool identifies words such as internet/network and communication/connectivity in the user' speech, it may guide the computer vision tool to look for LAN sockets or cables, Wi-Fi antennas and/or LEDs indications. Optionally, the keywords may be typed by agent 36p. Upon identifying the relevant objects in the image data 33i, the TSC system using the computer vision tool can analyze the object's setup/configuration and automatically identify potential issues/defects therein. Display device 36d may be used to present to the identified object to the agent. Once the issue is identified, agent 36p may instruct the user on how to solve it. If the solution is relatively simple, (e.g., press the power switch), agent 36p may provide verbal instructions. If user 33 is unable to carry out the verbal instructions, or the instructions are relatively complex, agent 36p may generate an instructive augmented reality video stream using one or more markers 39 and trackers (step S8). The markers are superimposed onto the image data and displayed on the user's mobile device in real time to provide additional guidance. The agent may alternatively superimpose annotations as discussed and described above with respect to FIGS. 1A-1C. Optionally, TSC 36 may query database 36r for a best working solution (step S7), based on the object's determined issues/defects, and transmits the best working solution to user 33. The instructions may comprise textual, auditory and/or annotated/augmented content. Agent 36p may provide user 33 with some (including one) or all types of instructive content, and/or limit the content to include some or the entire set of instructions." (Amir Par. 0066-0068).
receive a viewfinder frame from an electronic device associated with the user subsequent to initializing an AR session by the user in response to executing a first instruction from the set of instructions; and "Upon establishing a video support session, the support center processes and analyzes the sounds and image data received from the remote end user. The support center provides tools for conveying instructions to the remote user. For example, where image data is comprised of one or more still images, the support center may add annotations, e.g., text, signs and/or symbols to the image data. Where image data is comprised of real-time video stream or video frames, the support center may superimpose a movable augmented indicator onto the image data. The annotated or superimposed image data is presented on the display of the user's mobile device. When the remote user successfully resolves the problem by following the annotated/superimposed instructions, the problem and solution is stored in a cloud or other database record system. By storing various problems and solutions, a database of working solutions is gradually established. The database may be used by the support center to more quickly and efficiently solve future problems. Alternatively, the database may form an artificial intelligence, whereby the artificial intelligence and not an agent of the support center solves the problem using the image data and relays the instructions to the user using annotations or a moveable augmented indicator as described above" (Amir Par. 0046).
iteratively perform a plurality of operations until “FIG. 5B illustrates a decision tree used by support system 50 for determining the sequential instructions required to remedy the inoperability of an appliance. The input for the decision tree may be data derived from image data captured by image sensor 31c. For example, the image data may depict a current state of the inoperative product. The nodes in FIG. 5B represent operative states of the inoperative product and the lines represent the actions or the steps the user needs to complete to render the inoperative product operative.” (Amir Par. 0091). “If it is determined in step 50 that the best past solution obtained in steps 48-49 resolved the user's problem, a new database record 51 is constructed in step 54, and then stored in the database of the system for use in future trouble shooting sessions. The new database record may include one or more annotated images, a video showing how to fix the problem (with or without AR markers), text and/or audible instructions. If the best past solution is unsuccessful, other high-ranking solutions are obtained from the database, and presented in attempt to resolve the problem. Steps 48 to 50 may be repeated for each solution until a successful solution is found. Alternatively, or concurrently, speech analysis 12s, image recognition 12i, and steps 41 to 46 can be carried out to determine alternative problems/defects in the object 33e. (Amir Par. 0083).
electronically analyze the viewfinder frame to determine a subsequent instruction to be executed by the user from the set of instructions, “FIGS. 8A-8D illustrate another application of image processing system 60, wherein the image processing system 60 employs artificial intelligence during technical support. The image processing system comprises at least one processor configured to: receive image data captured by an image sensor of a mobile device, the image data including images of an inoperative appliance in an environment of a user; perform image recognition on the image data to determine an identity of the inoperative appliance and a likely cause of inoperability; retrieve a plurality of sequential instructions to be provided for enabling a user to complete a plurality of sequential actions in order to remedy the inoperability; cause the mobile device to sequentially display the plurality of sequential instructions; detect that the inoperative appliance is outside a field of view of the image sensor, based on the image data and during execution of the sequential actions; suspend display of additional sequential instructions while the inoperative appliance is outside of the field of view; detect when the inoperative appliance returns to the field of view after suspending display; and resume display of the additional sequential instructions after the inoperative appliance is detected to return to the field of view.” (Amir Par. 0122).
facilitate a display of an AR image frame on the electronic device, wherein the AR image frame is generated based, at least in part, on the subsequent instruction, “As seen in FIG. 8A, once an appliance 83e and relevant functional elements 83f, 83h, and 83i are detected and identified in the live video stream, and a likely source of inoperability is assessed, a plurality of sequential instructions 85 for repairing/assembling the object are displayed on the mobile device. A likely source of error may be derived from database 36r as discussed above, or deduced from the operational state of the elements. For example, if the processor detects, using the data processing techniques described above, that the appliance is not connected to a power source, it may deduce that a lack of power is the cause of inoperability. Instructions 85 may be displayed one at a time, or multiple instructions may be displayed simultaneously. FIG. 8A illustrates yet another embodiment wherein at least two instructions and a moveable augmented indicator 84a are simultaneously displayed on a mobile device 81.” (Amir Par. 0123).
determine an execution status of the subsequent instruction by monitoring the user while the user executes the subsequent instruction, the execution status indicating whether the subsequent instruction is one of successful and unsuccessful, and “Once retrieved, the processor annotates the image with one or more instructions for correcting the source of inoperability. FIG. 13B illustrates an annotated image wherein functional elements 103k, 103l have been identified as being in the wrong port. Annotation 134 therefore directs the user to switch the position of elements 103k and 103l to correct the configuration. Once corrected, the user may submit another image of the configuration to the processor, and the processor may notify the user that the step was completed successfully using positive feedback, e.g. check mark 135 (FIG. 13C).” (Amir Par. 0147). “The new database record may include one or more annotated images, a video showing how to fix the problem (with or without AR markers), text and/or audible instructions. If the best past solution is unsuccessful, other high-ranking solutions are obtained from the database, and presented in attempt to resolve the problem. Steps 48 to 50 may be repeated for each solution until a successful solution is found” (Amir Par. 0083).
Amir discloses facilitating an interaction between a user and an agent upon receiving a request for initiating an interaction from the user, receiving an augmented reality based workflow comprising a set of instructions from the agent, wherein the agent selects the AR-based workflow from a plurality of AR-based workflows based on interpreting a user objective for initiating the interaction, receiving a viewfinder frame from an electronic device associated with the user subsequent to initializing an AR session by the user in response to executing a first instruction from the set of instructions, iteratively performing a plurality of operations until each instruction from the set of instructions is executed, the plurality of operations comprising electronically analyzing the viewfinder frame to determine a subsequent instruction to be executed by the user from the set of instructions, facilitating a display of an AR image frame on the electronic device, wherein the AR image frame is generated based, at least in part, on the subsequent instruction, and determining an execution status of the subsequent instruction by monitoring the user while the user executes the subsequent instruction, the execution status indicating whether the subsequent instruction is one of successful and unsuccessful. Amir fails to disclose transmitting a notification indicating the execution status to the agent and wherein viewfinder frames are neither stored nor forwarded to the agent to ensure that the user's privacy is not compromised. Avila, however, does disclose:
transmit a notification indicating the execution status to the agent. “The routing determination component 114 is configured to output a routing determination in accordance with a state of the help desk case. For instance, in an example where a help desk request is being created, an interaction between the matched support agent and the customer may be initiated through a modality of the help desk service. In an example, where a first support agent is involved in a communication with a customer, a second support agent (e.g. matched support agent) may be added/patched into the communication. In examples where a routing determination relates to generation of a follow-up inquiry for an unresolved help desk case, the routing determination component 114 may automatically provide a notification to a support agent to follow-up with a customer. For instance, an agent may be automatically assigned to follow-up with a customer for an unresolved request. In another instance where an agent is assigned to a help desk case, an agent may automatically receive a reminder to follow-up with a customer. In further examples, a follow-up inquiry may automatically be transmitted based to a customer and/or support agent. For instance, the routing determination model may identify that a follow-up is needed for a help desk case, evaluate presence information for the customer and/or support agent and transmit a communication. This processing may be useful in keeping resolution of the help desk case as a priority” (Avila Par. 0038).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir with transmitting a notification indicating the execution status to the Agent of Avila to identify that a follow-up is needed for a help desk case, and keep resolution of a case as a priority (Avila Par. 0038).
The combination of Amir and Avila fail to disclose wherein viewfinder frames are neither stored nor forwarded to the agent to ensure that the user's privacy is not compromised. Chachek, however, does disclose augmented reality navigating a location. Chachek discloses,
wherein viewfinder frames are neither stored nor forwarded to the agent to ensure that the user's privacy is not compromised. “In some embodiments, some or most of all of the computer vision analysis of captured image footage or video footage, may be performed locally within the end-user device of each such user of the crowd-sourced participants; thereby reducing the need and the cost to send and receive large amounts of image data or video data, or to store or process them at cloud-based servers or store-located servers; and/or increasing the level of privacy that is provided to users, as the system need not continuously share or upload a live stream of video or images, which involves a greater exposure to the privacy of the user himself as well as other in-store customers around him, but rather, the system only selectively uploads particular images or image-portions that depict a recognized product, rather than images of faces or people); and/or also increasing the speed of detecting products and localizing items within the store due to parallel computing or distributed computing efforts (e.g., ten end-user devices are performing locally and in parallel in near-real-time their computer vision analysis on their ten respective images or video streams; instead of the latency that would be created by each device having to upload its images or videos to a processing queue at a remote server)” (Chachek Par. 0012).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir and Avila with neither storing nor forwarding data to ensure that the user's privacy is not compromised of Chachek to maintain privacy and/or anonymity of users (Chachek Par. 0177).
Regarding Claim 5, and Claim 14
The combination of Amir, Avila, and Chachek disclose the method of claim 1, as shown above. Amir, further discloses, The computer-implemented method of claim 1, wherein electronically analyzing the viewfinder frame further comprises: accessing, by the processing system, a plurality of images from a database associated with the processing system; “In some embodiments, TSC 36 is configured to record the video support sessions 21 in a database 36r. This way, the TSC 36 builds a continuously growing audio/visual database of user problems, and of their corresponding working solutions. The database may be used by computer vision tools to facilitate resolving of user's problems in future technical support sessions. Optionally, database 36r may be stored in a network computer/server of the TSC 36 (e.g., in the cloud)” (Amir Par. 0062). “Optionally, TSC 36 may query database 36r for a best working solution (step S7), based on the object's determined issues/defects, and transmits the best working solution to user 33” (Amir Par. 0068).
comparing, by the processing system, the viewfinder frame with the plurality of images; and “Control unit 12 is configured and operable to use image recognition module 12i to identify an object/functional element's set up/configuration and detect potential problems or defects therein. Database 36r can be used to store a plurality of erroneous setups/configurations (also referred to herein as reference data) to be compared by a comparison module 12u of control unit 12. Whenever the comparison module 12u determines that newly acquired image data 33i contains the same objects, issues or defects as the reference data, control unit 12 generates a diagnosis 12d identifying the erroneous setup/configuration identified in the image data 33i” (Amir Par. 0076).
upon determining a match between the viewfinder and an image from the plurality of images, determining, by the processing system, the subsequent instruction from the set of instructions based, at least in part, on the image. “Comparing new problems against a database of potentially related problems allows for precise problem identification. Once problems/defects are determined, database 36r is queried in step 48 for the best past solution as discussed above with respect to FIG. 2. After determining the best past solution, it is presented to the user via a display in the mobile device in step 49. The best past solution may include an annotated image, a video showing how to achieve the best problem solution (with or without AR markings), text and/or audible instructions, and is presented to the user via the display of the mobile device 31” (Amir Par. 0081).
Regarding Claim 8, and Claim 17
The combination of Amir, Avila, and Chachek disclose the method of claim 1, as shown above. Amir, further discloses, The computer-implemented method of claim 1, further comprising: upon determining that the execution status is unsuccessful generating, by the processing system, a set of intermediary instructions for rectifying the unsuccessful execution of the subsequent instruction; and “If the instructions are unsuccessful the user may be presented with an option to try again, or to receive human technical support. In the latter case, the processor may be configured to initiate a support session with a human operator when the processor determines that the completion of the plurality of sequential actions failed, or request that a technician be dispatched” (Amir Par. 0150).
facilitating, by the processing system, a display of another AR image frame on the electronic device, wherein the another AR image frame is generated based, at least in part, on the set of intermediary instructions. “The new database record may include one or more annotated images, a video showing how to fix the problem (with or without AR markers), text and/or audible instructions. If the best past solution is unsuccessful, other high-ranking solutions are obtained from the database, and presented in attempt to resolve the problem. Steps 48 to 50 may be repeated for each solution until a successful solution is found. Alternatively, or concurrently, speech analysis 12s, image recognition 12i, and steps 41 to 46 can be carried out to determine alternative problems/defects in the object 33e. If the problem is not resolved using a predefined number of database solutions, agent 36p may provide supplemental instructions or send a professional expert to the user 33 in step 52” (Amir Par. 0083).
Regarding Claim 9,
The combination of Amir, Avila, and Chachek disclose the method of claim 1, as shown above. Amir, further discloses, The computer-implemented method of claim 1, wherein the agent comprises at least one of a human and a virtual agent. “System 50 can be thus configured to concurrently conduct a plurality of support sessions 20, without any human intervention, using combined speech and image/video recognition techniques, to extract the proper and relevant keywords from auditory signals and/or image data obtained from user 33 that describe the experienced problem, and to determine the setup/configuration of the user's object” (Amir Par. 0100). “If the instructions are unsuccessful the user may be presented with an option to try again, or to receive human technical support. In the latter case, the processor may be configured to initiate a support session with a human operator when the processor determines that the completion of the plurality of sequential actions failed, or request that a technician be dispatched” (Amir Par. 0150).
Claim(s) 2, 3, 11, 12, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Amir (US 20210174371 A1), in view of Avila (US 20180278750 A1), in view of Chachek (US 20200302510 A1), and in further view of Waicberg (US 20210271882 A1) .
Regarding Claim 2, and Claim 11
The combination of Amir, Avila, and Chachek disclose the method of claim 1, as shown above. The combination of Amir, Avila, and Chachek fail to disclose a display of an option on the electronic device, the option enabling the user to initialize the AR session. Waicberg, however, does disclose The computer-implemented method of claim 1, further comprising: facilitating, by the processing system, a display of an option on the electronic device, the option enabling the user to initialize the AR session. “FIG. 5 details an example method 500 for providing an interactive troubleshooting call, according to some embodiments. Starting at step 505, a local user initiates a call to a remote expert via an application on the user's device. For example, in some embodiments, the user may use an interface similar to the one shown in FIG. 3 to select one or more remote users to initiate a session. For the purposes of this example, it will be assumed that the session is conducted between a single local user and a single remote user. However, in other embodiments, the AR-Based Service Platform may allow more than two users to simultaneously communicate via a session” (Waicberg Par. 0053).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir, Avila, and Chachek with a display of an option on the electronic device, the option enabling the user to initialize the AR session of Waicberg to help identify and resolve problems (Waicberg Par. 0044).
Regarding Claim 3, Claim 12, and Claim 19,
The combination of Amir, Avila, and Chachek disclose the method of claim 1, as shown above. Amir further discloses, The computer-implemented method of claim 1, further comprising: predicting, by the processing system, a user intent and a user persona based, at least in part, on the user interaction data; “Yet another aspect of the disclosure provides a non-transitory computer readable medium for assisting a user during technical support, the computer readable medium including instructions that, when executed by at least one processor, cause the at least one processor to perform a method, the method comprising: receiving at least one first image of an inoperative product captured by a mobile device of a user; performing image analysis on the at least one first image to identify in the at least one first image a status of a functional element associated with the inoperative product; accessing memory to determine a reason why the product is inoperative based on the determined status of the functional element; causing visual guidance to be displayed by the mobile device, wherein the visual guidance is associated with a plurality of sequential actions for causing the inoperative product to become operative; receiving at least one second image of the product, the second image being indicative of a completion of the plurality of sequential actions; performing image analysis on the at least one second image to determine that the completion of the plurality of sequential actions caused the inoperative product to become operative; and providing a notification to the user indicating problem resolution” (Amir Par. 0014).
Amir discloses a user intent and a user persona based on the user interaction data, but fails to disclose selecting the agent from a repository of agents based on the user intent and the user persona and accessing user interaction data from a database associated with the processing system, the user interaction data indicating information related to present and historical user interactions. Avila, however, does disclose:
selecting, by the processing system, the agent from a repository of agents based, at least in part, on the user intent and the user persona. “Moreover, generated insights may be useful in assisting support agents of the help desk in making additional routing determinations related to a help desk case. Examples of routing determinations that may be generated based on application of the routing determination model comprise but are not limited to: matching of a customer to a support agent, dedication of specific support agents to specific customers, determining a next step for resolution of an unresolved case (e.g. automatic resolution, follow-up inquiry, modality to use for a follow-up communication), evaluating when to contact a customer or identification of when a support agent is available based on presence information for customers and support agents and identification of predictive information that may be useful to provide to a support agent based on identification of issue/line of questioning by the customer, among other examples. For instance, evaluation of user-specific signal data may be used to determine whether to dedicate a specific support agent to a customer (e.g. for a predetermined period of time) in order to improve customer satisfaction. In such an example, user-specific signal data may be collected and analyzed in comparison with exemplary support agent data to determine a best match for dedicating a support agent to the customer. In another example, generated insights may be used to determine a next step in resolving an issue of the customer. For instance, evaluation of the user-signal data may yield an insight that a user prefers to receive communications through a specific modality (e.g. chat-based/instant messaging, phone-based, email, web-based platform, etc.) of the help desk service. As another example, an insight may be generated identifying that the customer prefers that an agent of a help desk attempt to automatically resolve an issue before contacting the customer. In such an instance, a matched support agent may be alerted to the fact that the user prefers an attempt at automatic resolution of their issue before receiving subsequent communication” (Avila Par. 0012).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir with selecting the agent from a repository of agents based on the user intent and the user persona of Avila to help identify and resolve problems (Avila Par. 0044).
The combination of Amir, Avila, and Chachek disclose a user intent and a user persona based on the user interaction data, and selecting the agent from a repository of agents based on the user intent and the user persona. The combination of Amir, Avila, and Chachek fail to disclose accessing user interaction data from a database associated with the processing system, the user interaction data indicating information related to present and historical user interactions. Waicberg, however, does disclose the following:
accessing, by the processing system, user interaction data from a database associated with the processing system, the user interaction data indicating information related to present and historical user interactions; “In the connected mode, the Client Application 185A allows an end user to reach out to an expert who can be another user in the field or an expert working a designated service location (e.g., a back office). In this scenario, images captured by the camera of the User Device 185 are streamed via the Client Application 185A to the other user. The other user is assumed to also be operating a device that executes an application that allows access to the AR-Based Service Platform 100. Once connected, either user can make annotations on the images and the annotated images are stored as discussed above. Additionally, the Audio/Video Recording Microservice 140 may be used to record any audio spoken by the users while performing the annotations, as well as the video stream showing how the annotations were applied. The AR-Based Service Platform 100 additionally includes a Machine Learning Microservice 180 that offers access to various machine learning algorithms. Such algorithms may be employed, for example, to create automated responses to user inquiries based on past session data” (Waicberg Par. 0043-0044).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir, Avila, and Chachek with accessing user related to present and historical user interactions of Waicberg in order to resolve issues faster without consulting a live human expert (Waicberg Par. 0031).
Claim(s) 4, 13, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Amir (US 20210174371 A1), in view of Avila (US 20180278750 A1), and in further view of Batlle (US 10264124 B2) .
Regarding Claim 4, Claim 13, and Claim 20,
The combination of Amir, Avila, and Chachek disclose the method of claim 1, as shown above. The combination of Amir, Avila, and Chachek fail to disclose accessing user relation management data and user interaction data from a database associated with the processing system, the user relation management data comprising information related to an electronic device type, a user location, and a user demographic and customizing, by the processing system, each instruction of the set of instructions based on the user relationship management data and the user interaction data. Batlle, however, does disclose, The computer-implemented method of claim 1, further comprising: accessing, by the processing system, user interaction data from a database associated with the processing system, “The method 700 proceeds to decision block 710 where the web server determines whether there is any user interaction with the customized website. In an embodiment, the web server 600 may determine whether the user is interacting with the customized website. If the user is no longer interacting or has closed out of the customized website, then the method 700 may end and the web server 600 may provide the information to the predictive management device 200. If the user interacts with the customized website, the method 700 may then proceed to block 712 where the web server transmits data associated with the interaction to the predictive management device 200. In an embodiment, the web server 600 may transmit any interactions by the user with the website hosted by the web server 600 to the predictive management device 200. The interactions may be transmitted as second data that the predictive management device 200 receives at block 312 of method 300 described above. The predictive management device 200 may determine second instructions based on the second data according to method 300, and transmit the second instructions to the user interaction database 114 or the web server 600 directly. The web server 600 may update the customized website based on the second instructions and provide the updated customized website to the user device 104 to be displayed on the display of the user device 104.” (Batlle Col. 15 Lines 15-26).
[accessing] user relation management data ,the user relation management data comprising information related to an electronic device type, a user location, and a user demographic; and “In a specific example, the contact center engine 1204 may be software or instructions stored on a computer-readable medium that is configured to gather frontend data about users interacting with the contact center system 1200 and/or customer service representatives interaction with the users via the customer service terminal 124 of FIG. 1 (e.g., data inputted by the user, data regarding the user device 104, location of the user, identification information, data inputs from a customer service representative via a customer service terminal 124, customer service representative specialties, customer service representative status, display rates of suggestions provided to customer service representatives, acceptance rates of suggestions, amount of time a customer service representative spends on a user interaction, if the customer service representative opts out of a suggestion, if the user issue is resolved, and other user and/or customer service representative related data that contact center system may gather known in the art). In a specific example, the contact center engine 1204 may be software or instructions stored on a computer-readable medium that is configured to generate and provide a customized customer service terminal GUI to be presented at a customer service terminal (e.g., customer service terminal 124) based on instructions generated by a predictive management device (e.g., predictive management device 200) and provide any of the other functionality that is discussed below that one of skill in the art in possession of the present disclosure will recognize as computer functionality that may be enabled by the customer service engine 1204 as well” (Batlle Col. 20 Lines 32-61).
customizing, by the processing system, each instruction of the set of instructions based, at least in part, on the user relationship management data and the user interaction data “The predictive management device may include one or more machine learning algorithms that may generate and provide instructions to a customer interaction database based on the historical data, real-time data, and/or any other data generated by the system and received by the predictive management device” (Batlle Col. 3 Lines 36-41).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir, Avila, and Chachek with accessing user relation management data and user interaction data from a database related to an electronic device type, a user location, and a user demographic and customizing instruction of the set of instructions based on the user relationship management data and the user interaction data of Batlle to have customized user experiences which lead to better retention of customers and more efficient use of customer service system resources to reduce costs of the service provider (Batlle Col. 4 Lines 25-27).
Claim(s) 6, 7, 15 and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Amir (US 20210174371 A1), in view of Avila (US 20180278750 A1), in view of Chachek (US 20200302510 A1), and in further view of Mandya (US 20210256561 A1) .
Regarding Claim 6, and Claim 15,
The combination of Amir, Avila, and Chachek disclose the method of claim 1, as shown above. The combination of Amir, Avila, and Chachek fail to disclose dynamically determining a promotional content based on the viewer frame, and generating the AR image frame based on the promotional content. Mandya, however, does disclose, The computer-implemented method of claim 1, wherein generating the AR image frame further comprises: dynamically determining, by the processing system, a promotional content based, at least in part, on the viewer frame; and “FIG. 1 depicts an augmented reality presentation including a promotion indicator 108 presented via a mobile device 102, according to some embodiments. As a customer shops in a retail facility, his or her mobile device 102 can transmit information to a backend system associated with the retail facility. For example, in one embodiment, the customer may provide his or her permission via an application for the retailer to receive the information from his or her mobile device 102. In one embodiment, the information includes a user identifier associated with the customer (e.g., a “user” of the system) and a location indicator of the mobile device 102. The backend system identifies the customer based on the user identifier and looks up user data for the customer. The user data can include any desired information, such as transaction histories, browsing histories, user demographics, user preferences, dates associated with users (e.g., birthdays, anniversaries, subscription expiration dates, etc.), etc. The system selects a promotion for the user based on the user data. For example, based on the user data, the system can determine that the customer typically purchases a one unit of a soft drink when he or she visits the retail facility but has not done so recently. In this example, the system can select a promotion for the soft drink in an attempt to 1) encourage the customer to purchase the soft drink and 2) create a feeling of appreciation in the customer” (Mandya Par. 0013).
generating, by the processing system, the AR image frame based, at least in part, on the promotional content. “The promotion is presented to the customer via the promotion indicator 108 in an augmented reality (AR) presentation. In one embodiment, the system fixes the promotion indicator 108 to a specific location. That is, the system associates a location, for example within the retail facility, with the promotion indicator 108 associated with the promotion for the customer. When the customer orients his or her mobile device 102 toward the location, the mobile device 102 presents the promotion indicator 108 in an augmented reality (AR) presentation. As depicted in FIG. 1, the augmented reality presentation includes the promotion indicator 108 located on an AR image 106 of a table 104. The augmented reality presentation also includes a text portion 110 that provides a description of the promotion selected for the customer. In embodiments in which the augmented reality presentation includes the text portion 110, the text portion 110 may not be presented until the customer selects the promotion indicator 108 via the mobile device 102 (e.g., taps the promotion indicator 108)” (Mandya Par. 0014).
It would have been obvious to one of ordinary skill in the art at the time of the claimed invention to have combined the image recognition of image data to determine an identity of the inoperative product and cause of the issue by retrieving instructions and providing visual guidance of Amir, Avila, and Chachek with dynamically determining a promotional content based on the viewer frame and generating the AR image frame based on the promotional content of Mandya to allow the user to feel that they are appreciated by the retailer and/ or special as opposed to a nuisance (Mandya Par. 0011).
Regarding Claim 7, and Claim 16
The combination of Amir, Avila, Chachek, and Mandya disclose the method of claim 6 as shown above. Amir further discloses, The computer-implemented method of claim 6, wherein generating the AR image frame further comprises: overlaying, by the processing system, the subsequent instruction on the viewfinder frame. “Once the issue is identified, agent 36p may instruct the user on how to solve it. If the solution is relatively simple, (e.g., press the power switch), agent 36p may provide verbal instructions. If user 33 is unable to carry out the verbal instructions, or the instructions are relatively complex, agent 36p may generate an instructive augmented reality video stream using one or more markers 39 and trackers (step S8). The markers are superimposed onto the image data and displayed on the user's mobile device in real time to provide additional guidance. The agent may alternatively superimpose annotations as discussed and described above with respect to FIGS. 1A-1C” (Amir Par. 0067).
Response to Arguments
Applicant's arguments filed 11/17/2025 with respect to 35 U.S.C. § 101, have been fully considered but they are not persuasive. Applicant recites that the claims focus on improving the process of guided instruction and task execution in AR environments, not on organizing human activity per se. The Examiner respectfully disagrees. MPEP 2106.04(a)(2)(II) states that a claimed invention is directed to certain methods of organizing human activity if the identified claim elements contain limitations that encompass fundamental economic principles or practices, commercial or legal interactions, or managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions). The Examiner submits that the identified claim elements represent a series of rules or instructions that a person or persons, with or without the aid of a computer, would follow to facilitate an interaction between a user and an agent upon receiving a request for initiating an interaction from the user, receiving a workflow comprising a set of instructions from the agent wherein the agent selects the workflow from a plurality of workflows based on interpreting a user objective for initiating the interaction, receive a viewfinder frame associated with the user subsequent to initializing a session by the user in response to executing a first instruction from the set of instructions wherein the viewfinder frames are neither stored nor forwarded to the agent to ensure that the user’s privacy is not compromised, iteratively performing a plurality of operations until each instruction from the set of instructions is executed, the plurality of operations comprising: analyzing the viewfinder frame to determine a subsequent instruction to be executed by the user from the set of instructions, facilitating a display of an image frame wherein the image frame is generated based on the subsequent instruction, determine an execution status of the subsequent instruction by monitoring the user while the user executes the subsequent instruction, the execution status indicating whether the subsequent instruction is one of successful and unsuccessful, and transmitting a notification indicating the execution status to the agent. Applicant argues that the claims are analogous to those in McRO, Inc. dba Planet Blue v. Bandai Namco Games America Inc., 120USPQ2d 1091 (Fed. Cir. 2016). The argument is not persuasive because the patent at issue in McRO dealt with an improvement in computer-related technology: automatic lip synchronization and facial expression animation using specific computer-implemented rules. With regard to McRO, Inc. dba Planet Blue v. Bandai Namco Games America Inc., No. 2015-1080, 21 (Fed. Cir. 2016), the Court cited Diehr, as follows: “The claims in Diehr, in contrast, were patentable. The claims likewise ‘employed a ‘well-known’ mathematical equation.’ Alice, 134 S. Ct. at 2358 (quoting Diehr, MCRO, INC. v. BANDAI NAMCO GAMES AMERICA 21 450 U.S. at 177). A computer performed the calculations as part of a broader process for curing rubber, but “the process as a whole [did] not thereby become unpatentable subject matter.’ Diehr, 450 U.S. at 187. Instead, the Court looked to how the claims “used that equation in a process designed to solve a technological problem in ‘conventional industry practice.’’ Alice, 134 S. Ct. at 2358 (quoting Diehr, 450 U.S. at 178). When looked at as a whole, ‘the claims in Diehr were patent eligible because they improved an existing technological process, not because they were implemented on a computer.’ Alice, 134 S. Ct. at 2358.” McRO, pg. 21. (Emphasis added). “When looked at as a whole, claim 1 is directed to a patentable, technological improvement over the existing, manual 3-D animation techniques. The claim uses the limited rules in a process specifically designed to achieve an improved technological result in conventional industry practice. Alice, 134 S. Ct. at 2358 (citing Diehr, 450 U.S. at 177).” McRO, pg. 27. Therefore, a determination must be made as to the focus of the claim(s) and whether the claim(s) are drawn to an improvement in computer-related technology whether it be to the operation of a computer or a computer network per se or a set of rules that improve computer-related technology by allowing computer performance of a function not previously performable by a computer. Looking at the limitations of Applicant’s claimed invention there is no indication that the combination of elements improves the functioning of a computer or improves any other technology. Their collective functions merely provide conventional computer implementation. In other words, the claims simply require the performance of the abstract idea of facilitating customer-agent interactions on generic computer components using conventional computer activities and unlike McRO, they are not drawn to an improvement in computer-related technology.
Applicant argues that the claims integrate AR technology into a practical application by enabling real-time adaptive instruction and monitoring, and improves the technical field of AR workflow management by providing a solution to the problem of delivering context-sensitive, adaptive guidance and monitoring in real time. The Examiner respectfully disagrees. The context-sensitive, adaptive guidance and monitoring that the Applicant argues does not recite a technical solution to a technical problem. Here, the Applicant’s argued problem is not a technological problem caused by the technological environment to which the claims are confined (computing system). Further, the problem of delivering context-sensitive, adaptive guidance, and monitoring in real time was not a problem caused by the computer/processor that is involved in the process. At best, the problem(s) described in the as-filed disclosure are business problems. The fact that the action occurs “in real time” is simply a matter of when the analysis occurs, not how the computer technology is improved. Applicant cites DDR Holdings, LLC. v. Hotels.com, L.P., 773 F.3d 1245, 1259 (Fed. Cir. 2014). However, DDR Holdings, LLC. v. Hotels.com, L.P., 773 F.3d 1245, 1259 (Fed. Cir. 2014) (finding that claiming a website that retained the “look and feel” of a host webpage provided a technological solution to the problem of retention of website visitors by utilizing a website descriptor that emulated the “look and feel” of the host webpage, where the problem arose out of the internet and was thus a technical problem). Here, the Examiner cannot find, nor has the Applicant identified, any technological problem that was caused by the technological environment to which the claims are confined.
Applicant argues that the claims require real-time visual input analysis, adaptive instruction delivery, execution monitoring, and conditional feedback—features that are not well-understood, routine, or conventional in the field of AR workflow management. Applicant cites Bascom Global Internet Services, Inc. v. AT&T Mobility LLC, 827 F.3d 1341 (Fed. Cir. 2016) and alleges that the office action does not adequately support a finding that the recited steps are well-understood, routine, and conventional activity. However, the examiner respectfully disagrees. The rejection does not rely on a finding that the claims recite additional elements that are insignificant extra solution activity that are well-understood or conventional, therefore, there is no burden on the examiner to provide a factual determination. Furthermore, the applicant alleges that non-conventional and non-generic arrangement of known elements mirrors the technological advancement in Bascom, where an non-conventional arrangement of components provided a patent- eligible solution. However, the examiner does not find the argument persuasive for two reasons. First, the current scope of the claims does not provide the features which provide the purported improvement, it merely recites the abstract idea merely being applied to a computer, and generally linked to a particular technological environment, with the intended result of a lower latency. Secondly, according to MPEP 2106.06(b), the finding in Bascom have been found to present a close call in Step 2A, thus relying on the analysis in step 2b, which found that the elements in combination amounted to significantly more because of the non-conventional and non-generic arrangement that provided a technical improvement in the art. This is unlike the present invention, for the reasons stated above, both that it is an example of merely applying, and a general link (MPEP 2106.05(h)). In addition, the rejection does not rely on a finding that the claims recite additional elements that are insignificant extra solution activity that are well-understood or conventional. Therefore, none of the arguments have been found persuasive and the 101 rejection is upheld.
Applicant's arguments filed 11/17/2025 with respect to 35 U.S.C. § 103, have been fully considered but they are now moot in view of updated rejections above which now include Amir, Avila, and Chachek as a combination. Therefore, the 103 rejection is maintained.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Emily M Kraisinger whose telephone number is (703)756-4583. The examiner can normally be reached M-F 7:30 AM -4:30 PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jessica Lemieux can be reached at 571-270-3445. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.M.K./Examiner, Art Unit 3626
/JESSICA LEMIEUX/Supervisory Patent Examiner, Art Unit 3626
Prosecution Insights