DETAILED ACTION
Statement of claims
The present amended application include :
a. Claims 1, 10, 16 and 20 were amended.
Claims 1-20 remain pending in the application. Claims 1-20 are being considered on the merits.
Response to Arguments
Rejections of Claims under 35 U.S.C. $ 103
Applicant argues that:
“The Maity system does not allow the user to choose or select what information he/she wishes to copy/retain after the information is presented on the video.”, “Bai does not appear to disclose or suggest using a clipboard of the computing device”.
Examiner respectfully disagrees and submits that Maity teaches enabling a user to select and copy one or more characters of at least one of text or alphanumeric information e.g., see para 53, 58, “displaying the web pages and other information that can be displayed in the browser program 143”, web page contents” and “type of content being text” for “a browser running on the computing device 140, sends a KVM redirection webpage to the browser. The browser, under the instruction of the webpage, can retrieve the redirection images from the web server”, “image 157 having information of all the pixels in the first frame 152 “ in par 64 and 68, see FIG. 4. Thus, “retrieve the redirection images” for “image 157 having information” . Therefore these teachings of Maity suggests allow the user to choose or select what information he/she wishes to copy/retain after the information is presented on the video (emphasis added)..Applicant is reminded that rejections are based on references as a whole and not just the cited passages. Although the specified citations are representative of the teachings in the art and are applied to the specific limitations within the individual claim, other passages and figures may apply as well. It is respectfully requested from the applicant, in preparing the responses, to fully consider the references in entirety as potentially teaching all or part of the claimed invention, as well as the context of the passage as taught by the cited art or disclosed by the examiner.
Moreover, Bai discloses using a clipboard of the computing device e.g., para 48, wherein “if the media file 130 includes video data, a video grabber module may capture one or more frames of the video data. The computing device 102 may extract text from the screen capture (e.g., using OCR) and determine the selected text 144 based on the positional data and the extracted text”. Thus, “ a video grabber module may capture one or more frames “ include the clipboard.
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai to allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-20 are rejected under 35 U.S.C. 103 as being unpatentable over Maity et al. (US 2014/0281894, Maity hereinafter) in view of Bai et al. (US 2019/0114072, Bai hereinafter).
As to claim 1, Maity teaches a method for enabling a user to select and copy one or more characters of at least one of text or alphanumeric information (e.g., para [0067] and [0068] “image 157 having information of all the pixels in the first frame 152 “, “ a copy of the first frame “ for “any corresponding pixels in the first and second frames have different values”, and “parameters received by the web server can be (x, y, W, H), indicating that the position of a tile is at coordinates (x, y) on the video frame and that the tile has a width of W and a height of H. The width and height parameters are optional” in par [0070] .
Thus, the “image 157 having information”, “x, y, W, H” include the alphanumeric information ) appearing with a video image frame being displayed on a display of a client computing device (e.g., “140”, FIG. 4) , during a keyboard, video and mouse (KVM) session in which the client computing device is being used (e.g., see FIG. 4 , para 61 and 62, wherein “ the computing device 140. “, “ The KVM redirection module can receive redirected video frames from the redirection controller and send keyboard and mouse input signals to the I/O controller 116.” , “The computing device 140 includes a display device 147 and input devices such as a keyboard 148, a mouse 149, and a touch panel” and “keyboards, touch pads, mouse, microphones, display screens, touch screens” , “for the computing device 140”, “the browser program 143 initially displays a first redirection image 157 on the canvas 160” in para 52 and 71, see FIG. 5.
Thus, the “redirected video frames” coupled with the “keyboard and mouse input signals” for the “client computing device 140” coupled with “I/O devices”, “touch screens” and “the browser program 143” , therefore a keyboard, video and mouse (KVM) session in which the client computing device is being used would have been inherent), the method comprising:
accessing a KVM appliance (e.g., “120”, FIG. 4) using a client computing device (e.g., “computing device 140”, FIG. 4) being operated by a user, wherein the client computing device is running a KVM application to carry out the KVM session (e.g para 50, “the browser program 143 can serve as a KVM redirection client of the system 100 for communicating with the BMC 120” and “FIG. 4 schematically depicts the web-based KVM redirection according to one embodiment of the present disclosure. As shown in FIG. 4, a Web Socket connection 150 is established between the BMC and the computing device 140. The host computer 110 may include a video controller 117 that output video signals, and a redirection controller 118 as the I/O port for the video redirection. The KVM redirection module 122 of the BMC 120 in this embodiment is connected to the redirection controller 118 and the I/O controller 116” in para [0064].
Thus, the browser program 143 can serve as a KVM redirection client” represents a KVM application to carry out the KVM session) ;
using the KVM application to control the KVM appliance to communicate with a target computer (e.g., “110, FIG. 5, para [0063] FIG. 5 schematically depicts a web-based KVM redirection according to certain embodiments of the present disclosure. As shown, the host computer 110 has a video controller 117, which outputs video signals. For example, the video signals can be output to a display device for display. The video controller 117 is in communication with a redirection controller 118, which captures the video signals and generates video frames. The redirection controller subsequently sends the video frames to the KVM redirection module 122 of the BMC 120.. Also, see FIG. 5) ;
using the KVM appliance to supply a video frame received from the target computer to a display operatively associated with the client computing device (e.g., see FIG. 5, para [0067] The KVM redirection module 122 receives the first and second video frames 152, 154 from the redirection controller 118 of the host computer 110. .), the video frame containing pixels making up the video frame (e.g., para [0066] “each pixel of the video frame output by the video controller has a respective corresponding pixel on the canvas.” and “ the pixels in the first frame 152 “ in para 67) , the pixels in the video frame forming at least one text or alphanumeric character (e.g., para 69, “the pixel values of the first and second video frames pixel by pixel” and “coordinates (x, y) on the video frame” in para [0070] . Thus, the “pixel values” having “ x, y” include the alphanumeric character) ;
receiving an input from a user controllable control component of the client computing device (e.g., para [0066] Based on the configuration information input by a user”, “the user may desire to have the same resolution as the resolution of video frames output by the video controller” and “allowing the user of the computing device 140 to input or select the configuration information “, “keyboard action generates corresponding input data, which may include a character code or other representative value corresponding to the keyboard event” , “keyboard action generates corresponding input data, which may include a character code or other representative value corresponding to the keyboard event” in para 74, 91 and 95. Thus, receiving an input from a user controllable control component of the client computing device ) .
However, Maity does not explicitly teach the text or alphanumeric information within the video image frame, the input defining a portion of the video frame selected by the user after viewing the video frame which includes the at least one text or alphanumeric character to be converted into a text output; using an optical character recognition (OCR) software application to convert the at least one text or alphanumeric character selected by the user into the text output; and using a clipboard of the client computing device to copy the text output for subsequent use by the user.
Bai teaches one of text or alphanumeric information appearing within a video image frame being displayed on a display of a client computing device (e.g., see para 13, “displaying media that includes text.” and “determine the selected text 144” in para 52) , the input defining a portion of the video frame selected by the user after viewing the video frame (e.g., para 14, “ view a media file on a computing device having a touchscreen display”) , which includes the at least one text or alphanumeric character selected by the user to be converted into a text output (e.g., para 13 “ select text displayed by the media “ and “portions of text from the media file that were displayed” , “ translating the selected text from a first language to a second language” in para 18 and 19) using an optical character recognition (OCR) software application to convert the at least one text or alphanumeric character into the text output (e.g., para 18, “The extracted text images may use optical character recognition (OCR) “ and “translating the selected text from a first language to a second language”, “ translating the selected text from a source language to a target language and performing a search using the translated text” in para 19 and 45
Also, See FIG. 4, Para 54 and 55 “extract text from the screen captures (e.g., using OCR) and determine the selected text 144 based on the positional data of the gestures 402 and 404 and the extracted text”, “perform one or more actions based on the selected text 144 “, “ include the translation 210 corresponding to the selected text 144”,
Therefore the input defining a portion of the video frame selected by the user which includes the at least one text or alphanumeric character to be converted into a text output using an optical character recognition (OCR) software application to convert the at least one text or alphanumeric character into the text output ), and using a clipboard of the client computing device to copy the text output for subsequent use by the user (para 19, wherein “After one or more words have been extracted from the screen capture(s) ), “displaying the translated text” and “the media file 130 includes video data, a video grabber module may capture one or more frames of the video data. The computing device 102 may extract text from the screen capture (e.g., using OCR) and determine the selected text 144 based on the positional data and the extracted text.”, “The computing device 102 may perform one or more actions based on the selected text 144 and provide the results 152 in the window 208” in para 48 and 49, see FIG. 3
para 48, wherein “if the media file 130 includes video data, a video grabber module may capture one or more frames of the video data. The computing device 102 may extract text from the screen capture (e.g., using OCR) and determine the selected text 144 based on the positional data and the extracted text”. Thus, “ a video grabber module may capture one or more frames “ include the clipboard.
Therefore , wherein “computing device 102 “ coupled with “a video grabber module may capture one or more frames of the video data “, “computing device 102 may perform one or more actions” include using a clipboard of the client computing device to copy the text output for subsequent use by the user ) .
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai for selecting and copying one or more characters of at least one of text or alphanumeric information appearing within a video image frame being displayed on a display of a client computing device, receiving an input from a user controllable control component of the client computing device, the input defining a portion of the video frame selected by the user which includes the at least one text or alphanumeric character to be converted into a text output; using an optical character recognition (OCR) software application to convert the at least one text or alphanumeric character into the text output; and using the client computing device to copy the text output for subsequent use by the user in order allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to combine the teachings of Maity with those of Bai because both references are directed to related systems addressing similar technical problems within the same field and seek to improve system performance, reliability, and efficiency.
Maity et al. disclose enabling a user to select and copy one or more characters of at least one of text or alphanumeric information appearing within a video image frame being displayed on a display of a client computing device, during a keyboard, video and mouse (KVM) session in which the client computing device is being used while Bai et al. teach pinput defining a portion of the video frame selected by the user after viewing the video frame, which includes the at least one text or alphanumeric character selected by the user to be converted into a text output; using an optical character recognition (OCR) software application to convert the at least one text or alphanumeric character selected by the user into the text output.
Incorporating the teachings of Bai et al. into the system of Maity et al. would have been a predictable and logical modification, yielding improved operational robustness and efficiency without requiring undue experimentation.
Such a combination would merely involve the substitution or integration of known elements performing their established functions, as taught by Bai et al., into the system of Maity et al., consistent with design incentives and market demands for improved performance and scalability. Moreover, Bai et al. explicitly recognize benefits to “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22) or “ an efficient user-intention guided text extraction algorithm to automatically extract text” (see Bai para 32) . —that would naturally be desirable in the system of Maity et al.
Accordingly, to one of ordinary skill in the art would have had a reasonable expectation of success in combining Maity et al. with Bai et al., and the combination represents no more than the predictable use of prior art elements according to their known functions.
As to claim 2, Maity does not teach wherein the OCR software application is running on the client computing device. However, BAI teaches wherein the OCR software application is running on the client computing device (e.g., see FIG. 2, para “ [0033] The text recognition module 122 may take the extracted text image 142 as input, and generate the selected text 144 using OCR. The text recognition module 122 may correlate the positional data 140 with the screen captures 132 to identify the text selected by the user, e.g., selected text 144. ).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai to allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
As to claim 3, Maity teaches further wherein the receiving an input from a user controllable control component comprises receiving an input from a user controlled touchpad of the client computing device ([0052] Further, the computing device 140 has at least one I/O device 146. Examples of the I/O device 146 include keyboards, touch pads”.) . However, Maity does not teach , to enable the user to highlight a portion of the video frame in which the at least one text or alphanumeric character appears. Bai teaches wherein the touchpad is configured to enable the user to highlight a portion of the video frame (see FIG. 2) in which the at least one text or alphanumeric character appears (e.g., see FIG. 2, para 40, “user may highlight keywords in the media file 130 “).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai to allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
As to claim 4, Maity teaches further wherein the receiving an input from a user controllable control component comprises receiving an input from an external mouse coupled to the client computing device (e.g., para [0092] A mouse can perform a variety of different actions, such as positioning the mouse cursor over a certain area, moving the mouse cursor, rolling the mouse wheel, and clicking (single click or double click) the mouse buttons.). However, Maity does not teach wherein the external mouse is configured to enable the user to highlight a portion of the video frame in which the at least one text or alphanumeric character appears. Bai teaches wherein the external mouse is configured to enable the user to highlight a portion of the video frame in which the at least one text or alphanumeric character appears (see FIG. 2, rejection of claim 2).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai to allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
As to claim 5, Maity does not teach wherein the text output comprises an ASCII text output. However Bai teaches wherein the text output comprises an ASCII text output (see FIG. 2).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai to allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
As to claim 6, Maity does not teach wherein the user controllable control component is configured to enable the user to copy the text output to a clipboard of an application running on the client computing device after conversion by the OCR software application. However Bai teaches wherein the user controllable control component is configured to enable the user to copy the text output to a clipboard (e.g., “a video grabber module “) of an application running on the client computing device after conversion by the OCR software application (e.g., para 48, wherein “if the media file 130 includes video data, a video grabber module may capture one or more frames of the video data. The computing device 102 may extract text from the screen capture (e.g., using OCR) and determine the selected text 144 based on the positional data and the extracted text”. Thus, “ a video grabber module may capture one or more frames “ include a clipboard of an application running, therefore wherein the user controllable control component is configured to enable the user to copy the text output to a clipboard of an application running on the client computing device after conversion by the OCR software application).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai to allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
As to claim 7, Maity teaches wherein the using the user controllable control component comprises using a touchpad operatively associated with the client computing device (e.g., para [[0052] Further, the computing device 140 has at least one I/O device 146. Examples of the I/O device 146 include keyboards, touch pads, mouse, microphones, display screens, touch screens, or other I/O devices applicable for the computing device 140. Some I/O devices, such as touch screens, are provided for the dual input/output purposes for the computing device 140..).
As to claim 8, Maity teaches wherein the using the user controllable control component comprises using an external mouse connected to the client computing device (para [0092] moving the mouse cursor, rolling the mouse wheel, and clicking (single click or double click) the mouse buttons.).
As to claim 9, Maity teaches wherein the accessing a KVM appliance using a client computing device comprises accessing the KVM appliance using one of: a laptop; a desktop computer; a computing tablet; or a smartphone (e.g., para [0048] the computing device 140 may include, for example, portable devices such as smartphones, tablets or other mobile computer devices)..
As to claim 10, see rejection of claim 1 above . Maity teaches further receiving a COPY command created using the client computing device (e.g., see FIG. 5, para 68, wherein “a copy of the first frame 152 kept “, “the web server 126 sends the patch to the browser program 143 on the computing device 140, and accordingly the browser program 143 can patch the canvas at the corresponding areas 170, 172.”. Thus, receiving a COPY command created using the client computing device would have been inherent) . However, Maity does not teach and in response to receiving the COPY command, copying the text output for subsequent use by the user. Bai teaches in response to receiving the COPY command, copying the text output for subsequent use by the user (e.g., “e.g., para [0031] The text extraction module 120 may extract text from the screen captures 132 as extracted text image 142. For example, the text extraction module 120 may use a user-intention guided text extraction algorithm to create extracted text image 142 from the screen captures 132. The text extraction module 120 may use the screen captures 132 and the user gesture 136 as input and output positions and bounding boxes of text areas, which may include user selected text, intended text (e.g., determining a user intention based on the user gesture and selecting text based on the user intention), and contextual text. “) .
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai to allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
As to claims 11-14, see rejection of claims 3-4, 6 and 9 above.
As to claim 15, Maity teaches further receiving a PASTE command initiated by the user from the control component (e.g., see FIG. 5, para 68, wherein “a copy of the first frame 152 kept “, “the web server 126 sends the patch to the browser program 143 on the computing device 140, and accordingly the browser program 143 can patch the canvas at the corresponding areas 170, 172.” Thus, receiving a PASTE command initiated by the user from the control component would have been inherent). However, Maity does not teach pasting the text output into at least one of a user selected application, a user selected document or a user selected web page. Bai teaches pasting the text output into at least one of a user selected application, a user selected document or a user selected web page (e.g., para [0031] The text extraction module 120 may extract text from the screen captures 132 as extracted text image 142. For example, the text extraction module 120 may use a user-intention guided text extraction algorithm to create extracted text image 142 from the screen captures 132. The text extraction module 120 may use the screen captures 132 and the user gesture 136 as input and output positions and bounding boxes of text areas, which may include user selected text, intended text (e.g., determining a user intention based on the user gesture and selecting text based on the user intention), and contextual text. For a tap and hold and drag while holding gesture, the text extraction module 120 may extract multiple lines of text from the image, including text that spans multiple frames of a video or multiple pages of a document. Thus, pasting the text output into at least one of a user selected application, a user selected document or a user selected web page would have been inherent).
Thus, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify the method of Maity by adopting the teachings of Bai to allow “a user viewing a technical presentation (e.g., a video, a set of slides, a document, web pages, etc.) may tap or swipe a technical word or phrase to obtain additional information associated with the technical word or phrase. ” (see Bai, para 22).
As to claim 16, see rejection of claims 1 and 5-6 and 10, 14-15.
As to claim 17-19, see rejection of claims 3, 6 and 9 above.
As to claim 20, see rejection of claim 1 above. Maity teaches further a system ,the system comprising: a computing device; a display operably associated with the computing device; a memory operably associated with the computing device, the memory configured to (see FIG. 1).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Vaindiner et al. discloses A system and method for accessing elements of a table in a digital image of the table, including: obtaining the digital image of the table; finding table elements in the digital image based on digital table properties, wherein the table elements define table cells; calculating coordinates of the table cells in the digital image based on the table elements; and accessing content of a selected table cell in the digital image using the coordinates of the selected table element.
Van discloses platform for recognising text using mobile devices with a built-in device video camera and automatically retrieving associated content based on the recognised text..
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ABDOU K SEYE whose telephone number is (571)270-1062. The examiner can normally be reached M-F 9-5:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Pierre Vital can be reached at 5712724215. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ABDOU K SEYE/Examiner, Art Unit 2198
/PIERRE VITAL/Supervisory Patent Examiner, Art Unit 2198
Prosecution Insights