aDETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1,11 and 18 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor, or for pre-AIA the applicant regards as the invention.
Regarding independent claim 1, the claim recites at least “ the user input commands”. The phrase “the” renders the claims indefinite because the claim has not mentioned user input commands earlier in the claim. For purpose of examination in view of the art below, the phrase “the user input commands " is interpreted as the user input.
Regarding independent claims 11 and 18, the claims recite similar limitations and are rejected under similar rationale.
Dependent claims 2-10, 12-17 and 19-20 inherit any deficiencies of the parent claim.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1-3, 5-13 and 15-20 are rejected under 35 U.S.C. 103 as being unpatentable over HAGHIGHAT (US 2016/0283055) in view of TODD (WO 2019 /191082 A2)
Regarding claim 1, HAGHIGHAT discloses: A computerized method for supplementing user interfacing and computer operations based on dynamic analysis of content, the method comprising: processing a plurality of user inputs via at least one of a plurality of input devices
(HAGHIGHAT; Figures 1-3; [0065] an input device processing component configured to process input for interaction with the contextual content interface component, wherein the input device processing component is adapted for processing one or more of mouse input, keyboard input, gesture input, video input, or audio input; [0047] A variety of selection mechanisms invoked from input device processing 525 … include selection through gestures; keyboard commands; speech commands; [0050], multiple computing devices; [0027], e.g. dynamic rendering and interaction with a designated website address 306 (e.g., a URL); )
and generating an output to the user based on the user input commands; (HAGHIGHAT , Figures 1-3; [0016] a content display user interface 102 ( e.g., a web browser software application) includes various interface commands 104 (e.g., a menu bar) that may be interacted with by a user for dynamic rendering and interaction, for the output of a designated website address 106 (e.g., a URL). The content display user interface 102 operates to render and generate an output screen 110 for the display of multimedia content (e.g., text, images, video) from the content retrieved from the designated website address 106; [0014] a user may be able to select the portion of the text "Joe's Chicago Pizza" and launch a contextual menu that provides selectable options [0015] generated contextual options; [0037] these context-based options may be provided from a listing of options provided in a contextual menu that may be
performed by user command in the graphical user interface. )
detecting a user capture of a portion of the output via further user input from one of the plurality of input devices, (HAGHIGHAT, Figures 1-3; Fig. 4A, e.g. item 402, “DETECT USER SELECTION OF CONTENT”; [0088] and capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116). The highlighted text portion 116 may be designated by the user with the use of a cursor selection, drag and highlight operation, gesture tap or swipe, or other user interface interaction; )
where the user capture generates an output display of a visual focus on the portion of the output; (HAGHIGHAT, Figures 1-3; [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116)...The highlighted text portion 116 may be expanded, contracted, or otherwise changed, moved, or re-focused on other portions of the displayed text content 114 based on additional user interaction commands; [0013], e.g. The contextual information interface described herein provides a mechanism to "lens-over" various webpage content [0088] outputting a plurality of context-based options for display in the graphical user interface, the plurality of context-based options designated to perform one or more actions in connection with the user selection of content; and capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; )
electronically processing the user capture to determine a content within the portion of the output within the visual focus; (HAGHIGHAT, [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 is generated for display and user interaction; [0013], e.g. The contextual information interface described herein provides a mechanism to "lens-over" various webpage content ... The contextual information interface provides custom contextual selectable actions and choices that will vary based on the type and characteristics of the content, as well as a semantic meaning of the content. [0088] capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; wherein the plurality of context-based options for operation on the selected content are determined from a context of the selected content indicated by an information service, wherein the plurality of context-based options are customized to a content type of the selected content and a user profile.)
dynamically generating a secondary display element visible to the user as part of the output, the secondary display element including a user interface display for receiving additional user input including at least one input command; (HAGHIGHAT, Figures 1-3, e.g. fig. 1, item 116, item 118; [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 (corresponding to “a secondary display element”) is generated for display and user interaction; [0019], e.g. The contextual information interface 118, however, may also include specific context- based options that change depending on the meaning of the selected text; [0033] The dynamic content options may be further customized and personalized; [0063],e.g. receive a user selection (corresponding to “additional user input”)of one of the plurality of context-based options to perform an associated action upon the selected content; [0075] outputting the plurality of context-based options in the user interface further including generating a display of the plurality of context-based options in a menu, wherein the user interface is a web browser configured to display one or more of text content, image content, or video content.)
in response to the additional user input, performing a processing operation in response to the at least one input command; (HAGHIGHAT, Figures 1-3, [0063],e.g. receive a user selection (corresponding to “additional user input”) of one of the plurality of context-based options to perform an associated action upon the selected content; [0033] The dynamic content options may be further customized and personalized not only based on user preferences and profiles, but also based on learning and user behaviors with the context interactive options;)
and generating an updated output for the user including an output of a result of the processing operations in response to the at least one input command. (HAGHIGHAT, Figures 1-3, [0033] The dynamic content options may be further customized and personalized not only based on user preferences and profiles, but also based on learning and user behaviors with the context interactive options; [0063],e.g. receive a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 is generated for display and user interaction;)
However HAGHIGHAT does not clearly disclose:
in response to the additional user input, accessing at least one artificial intelligence processing system for performing a processing operation in response to the at least one input command; and generating an updated output for the user including an output of a result of the processing operations as performed by the artificial intelligence processing system in response to the at least one input command.
However TODD discloses:
in response to the additional user input,
(TODD, [00569] an AI system 5402 may be used on or in connection with a feedback system, such as one that takes input from a user input device 3022 or from a system, such as a camera 5404 that captures feedback from a user, such as by gesture recognition; [0598] machine learning sub-systems for recognizing the initiation and cessation of an input session (such as by a special gesture, an audio signal, or the like). Techniques may include display of menus and choices for user hand gestures, such as a pop-up menu that takes focus and asks whether the user wants more information (e.g., allowing a thumbs up “yes”, thumbs down “no” or the like)
accessing at least one artificial intelligence processing system for performing a processing operation in response to the at least one input command; (TODD, [00569] an AI system 5402 may be used on or in connection with a feedback system, such as one that takes input from a user input device 3022 or from a system, such as a camera 5404 that captures feedback from a user, such as by gesture recognition; [00501] an IML session may ascertain commands from the text,…For example, saying “Ali Baba” may result in some personalized activity happening in the house, by establishing a connection between the audio and the voice command systems in the house; [00591] Output from an AI system 5402 may be used to trigger various actions, such as adding a content source 3010, highlighting an item [00217], e.g. The graphical user interface also controls the selection of content… in response to user commands; [00598]; [00568];)
and generating an updated output for the user including an output of a result of the processing operations as performed by the artificial intelligence processing system in response to the at least one input command. (TODD, [00591] Output from an AI system 5402 may be used to trigger various actions, such as adding a content source 3010, highlighting an item, selecting content, configuring content, moving to a location within content, finding content, configuring mixing and layering of content, changing detection modes within content, and the like; [00212] ,e.g. a video feed added in response to receipt of the command inputs; [00519] the consumer 4012 may ask the voice integration unit 4010,“What is the weather?” In response, the voice integration unit 4010 may respond with weather details. In one example, the voice integration unit 4010 may respond, “Weather is stormy, look at radar.” By way of this example, the video display system 46 may layer a signal over other signals being displayed on the video display device 38 to, therefore, display a moving color radar stream 4202 on the display 4004.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of HAGHIGHAT with the teaching of TODD in order to improve the relevance, personalization, and accuracy of contextual information presented to the user based on the selected content, (TODD, [00515], [00568], [00598]) and also to process relevant image portions faster and more accurate, (TODD, [00568]) and also to sensing, detecting, capturing, and/or responding to external events nearly simultaneously with their occurrence, or sufficiently fast to meet the expectations of the viewer, (TODD, [00428], [00579]).
Regarding claim 2, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 1 as outlined above. Claim 2 further recites: wherein the user capture of the at least a portion of the output includes a user input command via at least one the plurality of input devices to highlight at least a portion of text within the portion of the output display. (HAGHIGHAT, [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116). The highlighted text portion 116 may be designated by the user with the use of a cursor selection, drag and highlight operation, gesture tap or swipe, or other user interface interaction. The highlighted text portion 116 may be expanded, contracted, or otherwise changed, moved, or re-focused on other portions of the displayed text content 114 based on additional user interaction commands.)
Claims 12 and 19 correspond to claim 2, and are rejected accordingly.
Regarding claim 3, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 2 as outlined above. Claim 3 further recites: performing content recognition of the text within the portion to determine a contextual input; and performing the processing operation on the contextual input. (HAGHIGHAT, [0004] a graphical user interface enabling contextual selection and recognition of text content; [0016]-[0019]; [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116). [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the “context” of the content in the highlighted text portion), a contextual informationinterface 118 is generated for display and user interaction; [0019], e.g. The contextual information interface 118, however, may also include specific context- based options that change depending on the meaning of the selected text, such as "View Reviews for Car Model GX-200", "View 2014 Car Safety Test Results" (option 128), and "Locate a SportCar Dealer near Zip 98101" (option 130);)
Claim 13 corresponds to claim 3, and is rejected accordingly.
Regarding claim 5, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 3 as outlined above. Claim 5 further recites: accessing at least one preference database associated with the user; and acquiring a plurality of user preferences, wherein performing the processing operation includes generating personalized content based on the contextual input and the user preferences. (HAGHIGHAT, [0066] the contextual content component being operably coupled to a user profile data store, wherein the user profile data store provides the user profile, and wherein the user profile designates available actions upon the selected content based upon user demographics, user preferences, or prior user activity; [0033] The dynamic content options may be further customized and personalized not only based on user preferences and profiles, but also based on learning and user behaviors with the context interactive options. For example, the user behavior on which particular contextual options are accessed most often, and what type of content is generated with contextual options, may be tracked and captured in history. Other techniques may extend learning beyond what is presented to the user directly. )
Claim 15 corresponds to claim 5, and is rejected accordingly.
Regarding claim 6, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 1 as outlined above. HAGHIGHAT does not clearly disclose:
wherein the generating the updated output including inputting the output of the result of the processing operations into a first software application.
However TODD discloses:
wherein the generating the updated output including inputting the output of the result of the processing operations into a first software application. (TODD, [00500] an IML session may take feedback from audio content and feed it into a voice response system or voice integration unit, such as Alexa™; [00543], The results of an AI system 5402C 5402D may be sent to an intelligent layering system 5422 … and feed results to that AI system 5402D… and feed results to that AI system 5402D which may send the material to an intelligent layering system 5422 or on to another AI system 5402C)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of HAGHIGHAT with the teaching of TODD in order to improve the relevance, personalization, and accuracy of contextual information presented to the user based on the selected content, (TODD, [00515], [00568], [00598]) and also to process relevant image portions faster and more accurate, (TODD, [00568]) and also to sensing, detecting, capturing, and/or responding to external events nearly simultaneously with their occurrence, or sufficiently fast to meet the expectations of the viewer, (TODD, [00428], [00579]).
Regarding claim 7, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 1 as outlined above. Claim 7 further recites: wherein the user capture includes creating a visual outline around or encompassing at least one visual element. (HAGHIGHAT, Figures 1-3; [0028]-[0031]; [0028], e.g. the particular object may be selected with use of a cursor selection, highlight operation, gesture tap or drawing, or other user interface interaction; [0030] the designated portion 318 in the video content 314 is selected to indicate a particular portion of interest, here corresponding to an area around a representation of the person; [0013], e.g. The contextual information interface described herein provides a mechanism to "lens-over" various webpage content; [0029] The interaction may serve to select all or a portion of the video content 314. For example, only a designated portion 318 of objects depicted by the image content (e.g., the designated portion 318 representing a person in the video) may be selected by a user;)
Claim 16 corresponds to claim 7, and is rejected accordingly.
Regarding claim 8, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 7 as outlined above. Claim 8 further recites: performing content recognition of the visual element to determine a contextual input; and performing the processing operation on the contextual input. (HAGHIGHAT, Fig. 3, item 318, item 320; [0029]-[0031]; [0030] the designated portion 318 in the video content 314 is selected to indicate a particular portion of interest, here corresponding to an area around a representation of the person. The contextual information interface 320
may include application operations … change depending on a contextual meaning of the video content (e.g., the person depicted in designated portion 318 (the selected portion), the text depicted in selectable portion 316, and the like) such as “View PDF Brochure for GX-200” ... These operation choices may be determined by identifying the person represented in the designated portion 318, identifying the context of the selectable portion 316, and identifying the context of other video content depicted in video player 312; [0023] detection from recognized shapes and objects in the graphical content.)
Regarding claim 9, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 8 as outlined above. Claim 9 further recites: wherein the performing of the content recognition includes performing content recognition on at least a portion of text within the visual element. (HAGHIGHAT, [0004] FIG. 1 illustrates an overview of a graphical user interface enabling contextual selection and recognition of text content; [0029]-[0031]; [0031] For example, an image of the video content 314 is determined by an information source to represent an automobile, the selectable portion 316 is determined by an information source to refer to a text of specific model of an automobile,... The context-based options may also be determined based on other text, graphical content, or objects appearing in the video content 314, or with other text, graphical content, or objects displayed (or displayable) in the output screen 308.)
Regarding claim 10, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 7 as outlined above. Claim 10 further recites: wherein the visual element is at least one of: an image, a graph, and a graphic. (HAGHIGHAT, [0016]The content display user interface 102 operates to render and generate an output screen 110 for the display of multimedia content (e.g., text, images, video) from the content retrieved from the designated website address 106. Specifically, the content retrieved from the designated website address 106 includes image content 112 in addition to text content; [0023] The image content 212 is selected, in web browser, image viewer, or other content display interface with the use of a cursor selection, highlight operation, gesture tap or swipe, or other user interface interaction. The interaction may serve to select all or a portion of the image.)
Claim 17 corresponds to claim 10, and is rejected accordingly.
Regarding claim 11, HAGHIGHAT discloses: A computerized method for supplementing user interfacing and computer operations based on dynamic analysis of content, the method comprising: executing a first software application; (HAGHIGHAT, [0055] the software, when executed; [0016] a web browser software application; )
processing a plurality of user inputs to the first software application via at least one of a plurality of input devices (HAGHIGHAT; Figures 1-3; [0027], e.g. dynamic rendering and interaction with a designated website address 306 (e.g., a URL); [0016] a content display user interface 102 ( e.g., a web browser software application) includes various interface commands 104 (e.g., a menu bar) that may be interacted with by a user for dynamic rendering and interaction, for the output of a designated web site address 106 (e.g., a URL); [0065] an input device processing component configured to process input for interaction with the contextual content interface component, wherein the input device processing component is adapted for processing one or more of mouse input, keyboard input, gesture input, video input, or audio input; [0047] A variety of selection mechanisms invoked from input device processing 525 … include selection through gestures; keyboard commands; speech commands; [0050], multiple computing devices;)
and generating an output within the first software application to the user based on the user input commands; (HAGHIGHAT, Figures 1-3; [0016] a content display user interface 102 ( e.g., a web browser software application) includes various interface commands 104 (e.g., a menu bar) that may be interacted with by a user for dynamic rendering and interaction, for the output of a designated web site address 106 (e.g., a URL). The content display user interface 102 operates to render and generate an output screen 110 for the display of multimedia content (e.g., text, images, video) from the content retrieved from the designated website address 106; [0014] a user may be able to select the portion of the text "Joe's Chicago Pizza" and launch a contextual menu that provides selectable options [0015] generated contextual options; [0037] these context-based options may be provided from a listing of options provided in a contextual menu that may be performed by user command in the graphical user interface. )
detecting a user capture of a portion of the output from the first application via further user input from one of the plurality of input devices, (HAGHIGHAT, Figures 1-3; Fig. 4A, e.g. item 402, “DETECT USER SELECTION OF CONTENT”; [0088] and capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116). The highlighted text portion 116 may be designated by the user with the use of a cursor selection, drag and highlight operation, gesture tap or swipe, or other user interface interaction; )
where the user capture generates an output display of a visual focus on the portion of the output; (HAGHIGHAT, Figures 1-3; [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116)...The highlighted text portion 116 may be expanded, contracted, or otherwise changed, moved, or re-focused on other portions of the displayed text content 114 based on additional user interaction commands; [0013], e.g. The contextual information interface described herein provides a mechanism to "lens-over" various webpage content [0088] outputting a plurality of context-based options for display in the graphical user interface, the plurality of context-based options designated to perform one or more actions in connection with the user selection of content; and capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; )
electronically processing the user capture to determine a content within the portion of the output within the visual focus; (HAGHIGHAT, [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 is generated for display and user interaction; [0013], e.g. The contextual information interface described herein provides a mechanism to "lens-over" various webpage content ... The contextual information interface provides custom contextual selectable actions and choices that will vary based on the type and characteristics of the content, as well as a semantic meaning of the content. [0088] capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; wherein the plurality of context-based options for operation on the selected content are determined from a context of the selected content indicated by an information service, wherein the plurality of context-based options are customized to a content type of the selected content and a user profile.)
dynamically generating a secondary display element visible to the user as part of the output, the secondary display element including a user interface display for receiving additional user input including at least one input command; (HAGHIGHAT, Figures 1-3, e.g. fig. 1, item 116, item 118; [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 (corresponding to “a secondary display element”) is generated for display and user interaction; [0019], e.g. The contextual information interface 118, however, may also include specific context- based options that change depending on the meaning of the selected text; [0033] The dynamic content options may be further customized and personalized; [0063],e.g. receive a user selection (corresponding to “additional user input”)of one of the plurality of context-based options to perform an associated action upon the selected content; [0075] outputting the plurality of context-based options in the user interface further including generating a display of the plurality of context-based options in a menu, wherein the user interface is a web browser configured to display one or more of text content, image content, or video content.)
in response to the additional user input, performing a processing operation in response to the at least one input command; (HAGHIGHAT, Figures 1-3, [0063],e.g. receive a user selection (corresponding to “additional user input”) of one of the plurality of context-based options to perform an associated action upon the selected content; [0033] The dynamic content options may be further customized and personalized not only based on user preferences and profiles, but also based on learning and user behaviors with the context interactive options;)
and generating an updated output for the user including an output of a result of the processing operations in response to the at least one input command including generating a new output display for the user in a second application. (HAGHIGHAT, Figures 1-3, [0033] The dynamic content options may be further customized and personalized not only based on user preferences and profiles, but also based on learning and user behaviors with the context interactive options; [0063],e.g. receive a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 is generated for display and user interaction; [0030] The contextual information interface 320 however includes specific operations that change depending on a contextual meaning of the video content (e.g., the person depicted in designated portion 318 (the selected portion), the text depicted in selectable portion 316, and the like) such as "View PDF Brochure forGX-200" ( option 326),)
However HAGHIGHAT does not clearly disclose:
in response to the additional user input, accessing at least one artificial intelligence processing system for performing a processing operation in response to the at least one input command; and generating an updated output for the user including an output of a result of the processing operations as performed by the artificial intelligence processing system in response to the at least one input command including generating a new output display for the user in a second application.
However TODD discloses:
in response to the additional user input,
(TODD, [00569] an AI system 5402 may be used on or in connection with a feedback system, such as one that takes input from a user input device 3022 or from a system, such as a camera 5404 that captures feedback from a user, such as by gesture recognition; [0598] machine learning sub-systems for recognizing the initiation and cessation of an input session (such as by a special gesture, an audio signal, or the like). Techniques may include display of menus and choices for user hand gestures, such as a pop-up menu that takes focus and asks whether the user wants more information (e.g., allowing a thumbs up “yes”, thumbs down “no” or the like)
accessing at least one artificial intelligence processing system for performing a processing operation in response to the at least one input command; (TODD, [00569] an AI system 5402 may be used on or in connection with a feedback system, such as one that takes input from a user input device 3022 or from a system, such as a camera 5404 that captures feedback from a user, such as by gesture recognition; [00501] an IML session may ascertain commands from the text,…For example, saying “Ali Baba” may result in some personalized activity happening in the house, by establishing a connection between the audio and the voice command systems in the house; [00591] Output from an AI system 5402 may be used to trigger various actions, such as adding a content source 3010, highlighting an item [00217], e.g. The graphical user interface also controls the selection of content… in response to user commands; [00598]; [00568];)
and generating an updated output for the user including an output of a result of the processing operations as performed by the artificial intelligence processing system in response to the at least one input command (TODD, [00591] Output from an AI system 5402 may be used to trigger various actions, such as adding a content source 3010, highlighting an item, selecting content, configuring content, moving to a location within content, finding content, configuring mixing and layering of content, changing detection modes within content, and the like; [00212] ,e.g. a video feed added in response to receipt of the command inputs; [00519] the consumer 4012 may ask the voice integration unit 4010,“What is the weather?” In response, the voice integration unit 4010 may respond with weather details. In one example, the voice integration unit 4010 may respond, “Weather is stormy, look at radar.” By way of this example, the video display system 46 may layer a signal over other signals being displayed on the video display device 38 to, therefore, display a moving color radar stream 4202 on the display 4004.)
including generating a new output display for the user in a second application. (Figs 41-42; e.g. Fig. 41, Television including platform , Video plus independent HTML 5 web page overlay controllable via voice units and other voice integration (through platform APIs; [00518]-[00519]; [00519] the consumer 4012 may ask the voice integration unit 4010,“What is the weather?” In response, the voice integration unit 4010 may respond with weather details. In one example, the voice integration unit 4010 may respond, “Weather is stormy, look at radar.” By way of this example, the video display system 46 may layer a signal over other signals being displayed on the video display device 38 to, therefore, display a moving color radar stream 4202 on the display 4004.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of HAGHIGHAT with the teaching of TODD in order to improve the relevance, personalization, and accuracy of contextual information presented to the user based on the selected content, (TODD, [00515], [00568], [00598]) and also to process relevant image portions faster and more accurate, (TODD, [00568]) and also to sensing, detecting, capturing, and/or responding to external events nearly simultaneously with their occurrence, or sufficiently fast to meet the expectations of the viewer, (TODD, [00428], [00579]).
Regarding claim 18, HAGHIGHAT discloses: A computerized method for supplementing user interfacing and computer operations based on dynamic analysis of content, the method comprising: executing a first software application; (HAGHIGHAT, [0055] the software, when executed; [0016] a web browser software application; )
processing a plurality of user inputs to the first software application via at least one of a plurality of input devices (HAGHIGHAT; Figures 1-3; [0027], e.g. dynamic rendering and interaction with a designated website address 306 (e.g., a URL); [0016] a content display user interface 102 ( e.g., a web browser software application) includes various interface commands 104 (e.g., a menu bar) that may be interacted with by a user for dynamic rendering and interaction, for the output of a designated web site address 106 (e.g., a URL); [0065] an input device processing component configured to process input for interaction with the contextual content interface component, wherein the input device processing component is adapted for processing one or more of mouse input, keyboard input, gesture input, video input, or audio input; [0047] A variety of selection mechanisms invoked from input device processing 525 … include selection through gestures; keyboard commands; speech commands; [0050], multiple computing devices;)
and generating an output within the first software application to the user based on the user input commands; (HAGHIGHAT, Figures 1-3; [0016] a content display user interface 102 ( e.g., a web browser software application) includes various interface commands 104 (e.g., a menu bar) that may be interacted with by a user for dynamic rendering and interaction, for the output of a designated web site address 106 (e.g., a URL). The content display user interface 102 operates to render and generate an output screen 110 for the display of multimedia content (e.g., text, images, video) from the content retrieved from the designated website address 106; [0014] a user may be able to select the portion of the text "Joe's Chicago Pizza" and launch a contextual menu that provides selectable options [0015] generated contextual options; [0037] these context-based options may be provided from a listing of options provided in a contextual menu that may be performed by user command in the graphical user interface. )
detecting a user capture of a portion of the output from the first application via further user input from one of the plurality of input devices, (HAGHIGHAT, Figures 1-3; Fig. 4A, e.g. item 402, “DETECT USER SELECTION OF CONTENT”; [0088] and capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116). The highlighted text portion 116 may be designated by the user with the use of a cursor selection, drag and highlight operation, gesture tap or swipe, or other user interface interaction; )
where the user capture generates an output display of a visual focus on the portion of the output; (HAGHIGHAT, Figures 1-3; [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116)...The highlighted text portion 116 may be expanded, contracted, or otherwise changed, moved, or re-focused on other portions of the displayed text content 114 based on additional user interaction commands; [0013], e.g. The contextual information interface described herein provides a mechanism to "lens-over" various webpage content [0088] outputting a plurality of context-based options for display in the graphical user interface, the plurality of context-based options designated to perform one or more actions in connection with the user selection of content; and capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; )
electronically processing the user capture to determine a content within the portion of the output within the visual focus; (HAGHIGHAT, [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 is generated for display and user interaction; [0013], e.g. The contextual information interface described herein provides a mechanism to "lens-over" various webpage content ... The contextual information interface provides custom contextual selectable actions and choices that will vary based on the type and characteristics of the content, as well as a semantic meaning of the content. [0088] capturing input from a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; wherein the plurality of context-based options for operation on the selected content are determined from a context of the selected content indicated by an information service, wherein the plurality of context-based options are customized to a content type of the selected content and a user profile.)
dynamically generating a secondary display element visible to the user as part of the output, the secondary display element including a user interface display for receiving additional user input including at least one input command; (HAGHIGHAT, Figures 1-3, e.g. fig. 1, item 116, item 118; [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 (corresponding to “a secondary display element”) is generated for display and user interaction; [0019], e.g. The contextual information interface 118, however, may also include specific context- based options that change depending on the meaning of the selected text; [0033] The dynamic content options may be further customized and personalized; [0063],e.g. receive a user selection (corresponding to “additional user input”)of one of the plurality of context-based options to perform an associated action upon the selected content; [0075] outputting the plurality of context-based options in the user interface further including generating a display of the plurality of context-based options in a menu, wherein the user interface is a web browser configured to display one or more of text content, image content, or video content.)
performing content recognition of the text within the portion to determine a contextual input; performing the processing operation on the contextual input; (HAGHIGHAT, [0004] a graphical user interface enabling contextual selection and recognition of text content; [0016]-[0019]; [0017] Within the display of the text content 114, a text portion is selected by user interaction commands (indicated by the highlighted text portion 116). [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the “context” of the content in the highlighted text portion), a contextual informationinterface 118 is generated for display and user interaction; [0019], e.g. The contextual information interface 118, however, may also include specific context- based options that change depending on the meaning of the selected text, such as "View Reviews for Car Model GX-200", "View 2014 Car Safety Test Results" (option 128), and "Locate a SportCar Dealer near Zip 98101" (option 130);)
accessing at least one preference database associated with the user; acquiring a plurality of user preferences, wherein performing the processing operation includes generating personalized content based on the contextual input and the user preferences; (HAGHIGHAT, [0066] the contextual content component being operably coupled to a user profile data store, wherein the user profile data store provides the user profile, and wherein the user profile designates available actions upon the selected content based upon user demographics, user preferences, or prior user activity; [0033] The dynamic content options may be further customized and personalized not only based on user preferences and profiles, but also based on learning and user behaviors with the context interactive options. For example, the user behavior on which particular contextual options are accessed most often, and what type of content is generated with contextual options, may be tracked and captured in history. Other techniques may extend learning beyond what is presented to the user directly. )
in response to the additional user input, performing a processing operation in response to the at least one input command; (HAGHIGHAT, Figures 1-3, [0033] The dynamic content options may be further customized and personalized not only based on user preferences and profiles, but also based on learning and user behaviors with the context interactive options; [0063],e.g. receive a user selection of one of the plurality of context-based options to perform an associated action upon the selected content;)
and generating an updated output for the user including an output of a result of the processing operations in response to the at least one input command including generating a new output display for the user in at least one of: the first application and a second application. (HAGHIGHAT, Figures 1-3, [0033] The dynamic content options may be further customized and personalized not only based on user preferences and profiles, but also based on learning and user behaviors with the context interactive options; [0063],e.g. receive a user selection of one of the plurality of context-based options to perform an associated action upon the selected content; [0018] Based on the content and meaning of the highlighted text portion 116 (e.g., the "context" of the content in the highlighted text portion), a contextual information interface 118 is generated for display and user interaction; [0030] The contextual information interface 320 however includes specific operations that change depending on a contextual meaning of the video content (e.g., the person depicted in designated portion 318 (the selected portion), the text depicted in selectable portion 316, and the like) such as "View PDF Brochure forGX-200" ( option 326),)
However HAGHIGHAT does not clearly disclose:
in response to the additional user input, accessing at least one artificial intelligence processing system for performing a processing operation in response to the at least one input command; and generating an updated output for the user including an output of a result of the processing operations as performed by the artificial intelligence processing system in response to the at least one input command including generating a new output display for the user in at least one of: the first application and a second application.
However TODD discloses:
in response to the additional user input,
(TODD, [00569] an AI system 5402 may be used on or in connection with a feedback system, such as one that takes input from a user input device 3022 or from a system, such as a camera 5404 that captures feedback from a user, such as by gesture recognition; [0598] machine learning sub-systems for recognizing the initiation and cessation of an input session (such as by a special gesture, an audio signal, or the like). Techniques may include display of menus and choices for user hand gestures, such as a pop-up menu that takes focus and asks whether the user wants more information (e.g., allowing a thumbs up “yes”, thumbs down “no” or the like)
accessing at least one artificial intelligence processing system for performing a processing operation in response to the at least one input command (TODD, [00569] an AI system 5402 may be used on or in connection with a feedback system, such as one that takes input from a user input device 3022 or from a system, such as a camera 5404 that captures feedback from a user, such as by gesture recognition; [00501] an IML session may ascertain commands from the text,…For example, saying “Ali Baba” may result in some personalized activity happening in the house, by establishing a connection between the audio and the voice command systems in the house; [00591] Output from an AI system 5402 may be used to trigger various actions, such as adding a content source 3010, highlighting an item [00217], e.g. The graphical user interface also controls the selection of content… in response to user commands; [00598]; [00568];)
and generating an updated output for the user including an output of a result of the processing operations as performed by the artificial intelligence processing system in response to the at least one input command (TODD, [00591] Output from an AI system 5402 may be used to trigger various actions, such as adding a content source 3010, highlighting an item, selecting content, configuring content, moving to a location within content, finding content, configuring mixing and layering of content, changing detection modes within content, and the like; [00212] ,e.g. a video feed added in response to receipt of the command inputs; [00519] the consumer 4012 may ask the voice integration unit 4010,“What is the weather?” In response, the voice integration unit 4010 may respond with weather details. In one example, the voice integration unit 4010 may respond, “Weather is stormy, look at radar.” By way of this example, the video display system 46 may layer a signal over other signals being displayed on the video display device 38 to, therefore, display a moving color radar stream 4202 on the display 4004.)
including generating a new output display for the user in at least one of: the first application and a second application. (Figs 41-42; e.g. Fig. 41, Television including platform , Video plus independent HTML 5 web page overlay controllable via voice units and other voice integration (through platform APIs; [00518]-[00519]; [00519] the consumer 4012 may ask the voice integration unit 4010,“What is the weather?” In response, the voice integration unit 4010 may respond with weather details. In one example, the voice integration unit 4010 may respond, “Weather is stormy, look at radar.” By way of this example, the video display system 46 may layer a signal over other signals being displayed on the video display device 38 to, therefore, display a moving color radar stream 4202 on the display 4004.)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of HAGHIGHAT with the teaching of TODD in order to improve the relevance, personalization, and accuracy of contextual information presented to the user based on the selected content, (TODD, [00515], [00568], [00598]) and also to process relevant image portions faster and more accurate, (TODD, [00568]) and also to sensing, detecting, capturing, and/or responding to external events nearly simultaneously with their occurrence, or sufficiently fast to meet the expectations of the viewer, (TODD, [00428], [00579]).
Regarding claim 20, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 18 as outlined above. Claim 20 further recites: wherein the user capture includes creating a visual outline around or encompassing at least one visual element (HAGHIGHAT, Figures 1-3; [0028]-[0031]; [0028], e.g. the particular object may be selected with use of a cursor selection, highlight operation, gesture tap or drawing, or other user interface interaction; [0030] the designated portion 318 in the video content 314 is selected to indicate a particular portion of interest, here corresponding to an area around a representation of the person; [0013], e.g. The contextual information interface described herein provides a mechanism to "lens-over" various webpage content; [0029] The interaction may serve to select all or a portion of the video content 314. For example, only a designated portion 318 of objects depicted by the image content (e.g., the designated portion 318 representing a person in the video) may be selected by a user;)
and the visual element is at least one of: an image, a graph, and a graphic. (HAGHIGHAT, [0016]The content display user interface 102 operates to render and generate an output screen 110 for the display of multimedia content (e.g., text, images, video) from the content retrieved from the designated website address 106. Specifically, the content retrieved from the designated website address 106 includes image content 112 in addition to text content; [0023] The image content 212 is selected, in web browser, image viewer, or other content display interface with the use of a cursor selection, highlight operation, gesture tap or swipe, or other user interface interaction. The interaction may serve to select all or a portion of the image.)
Claims 4 and 14 are rejected under 35 U.S.C. 103 as being unpatentable over HAGHIGHAT (US 2016/0283055) in view of TODD (WO 2019 /191082 A2) in view of Nawrocki (US 11,128,680 B2)
Regarding claim 4, HAGHIGHAT in view of TODD discloses all of the features with respect to claim 3 as outlined above. HAGHIGHAT in view of TODD does not clearly disclose:
wherein the performing the processing operation includes generating a structured document therefrom.
However Nawrocki discloses:
wherein the performing the processing operation includes generating a structured document therefrom. (Nawrocki, column 1, line 62- evaluating and parsing communication streams and generating documents from the parsed communication streams that are formatted to satisfy specific corporate reporting requirement ;column 9, line 24- Participants can prompt the AI engine using particular keywords such as "Agenda change" which indicates that the committee is moving onto a next agenda item, "Executive Session", which temporarily turns of listening by the AI engine, or "AI engine, proceed" to resume AI engine listening. After having analyzed the data, in step 314, the Minute Maker module generates initial long and short of meeting Minutes ;column 7, line 46- The Minute Maker 206 employs an artificial intelligence engine (AI engine) to analyze received text and audio streams related to a particular meeting, and organizes the received data in a document referred to as the "Minutes" of the meeting;)
Therefore, it would have been obvious to a person of ordinary skill in the art before the effective filing date of the claimed invention to incorporate the teaching of HAGHIGHAT in view of TODD with the teaching of Nawrocki in order to classify the words and sentences in the audio stream and organize the textual data according to known templates, (Nawrocki, column 9, lines 21-23) and also to automation of certain fiduciary tasks using machine learning techniques, including generation of corporate meeting minute documents in compliance with organizational requirements, (Nawrocki, abstract).
Claim 14 corresponds to claim 4, and is rejected accordingly.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Faezeh Forouharnejad whose telephone number is (571)270-7416. The examiner can normally be reached on Mondays, Wednesdays and Thursdays.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Shah Sanjiv can be reached on (571)272-4098. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from Patent Center and the Private Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from Patent Center or Private PAIR. Status information for unpublished applications is available through Patent Center and Private PAIR to authorized users only. Should you have questions about access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free)
/F.F. /
Examiner, Art Unit 2166
/SANJIV SHAH/ Supervisory Patent Examiner, Art Unit 2166
