DETAILED ACTION
Receipt of Applicant’s Amendment, filed February 24, 2026 is acknowledged.
Claims 1, 9, and 17 were amended.
Claims 6 and 14 were cancelled.
Claims 1-5, 7-13, 15-20 are pending in this office action.
Claim Interpretation
Please note this claim interpretation section is provided for clarity of the record and should not be taken as indication of any issue within the present application.
With regard to claims 1, 3, 9, 11 and 16, claim 1 recites “identifying, based on the posit on of the matched area in the screenshot by the first terminal, a matched file icon from the plurality of file icons.” Claims 9 and 16 recite similar language. Claims 3 and 11 recite language impacted by this interpretation as well. The claim has previously recited that the picture (taken by and sent from the 2nd terminal) contains a “target file icon”. When read within light of the specification, it is clear that the matched file icon is intended to be the same icon as the ‘target file icon’ but is present within the screenshot of the first terminal. To be clear, both the screenshot and the picture contain an icon which appears the same. This icon is labeled “matched file icon” in the screenshot and “target file icon” in the picture.
It is also noted that the second terminal takes the picture, while the first terminal takes the screenshot. The picture is taken of the screen of the first terminal.
Claim Rejections - 35 USC § 103
The following is a quotation of pre-AIA 35 U.S.C. 103(a) which forms the basis for all obviousness rejections set forth in this Office action:
(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102, if the differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 1-3, 7-11, 15-18 are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Chang [2012/0070090] in view of Huijuan [Fast Image Matching Based-on Improved SURF Algorithm], Amanieux [2014/0250203], Bhatia [2012/0209922], and Stone [5467411].
With regard to claim 1 Chang teaches A method for transmitting (Chang, ¶5 “mechanisms for moving information conveniently between computing devices”) a target as the target of the image (Chang, ¶67 “In examples similar to those discussed above, the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”) file as the requested data may be a file (Chang, ¶57 “In some cases, an application can attach offline resources or files.”) between terminals (Chang, Figure 2A 204, 206, 208), the method comprising:
[a] Receiving (Chang, ¶41 “the computer 204 receives the message from computer 208”), by a first terminal as the Computer 204 (Chang, Figure 2A, 204; Figure 3A-B “Target Computer”; ¶32 “Three main components of the system 200 include a desktop computer 204, a computer 208 (which may be a mobile computer), and a server system 206.”) comprising as computer 204 may be a generic computing device 700 (Chang, ¶90 “FIG. 7 shows an example of a generic computer device 700 and a generic mobile computer device 750, which may be used with the techniques described here.”) a memory as memory 704 (Chang, ¶91 “The processor 702 can process instructions for execution within the computing device 700, including instructions stored in the memory 704”) storing instructions as instructions (Id) and a processor as processor 702 (Id) in communication with the memory (Id), a picture as the message contains the captured image (Chang, ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”) comprising at least a portion as at least a portion of the image (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”) of a screen as the image displayed on the monitory of computer 204 (Id) of the first terminal as computer 204 (Id) from as computer 208 captures the image and sends the message (Chang, ¶39 and ¶41) a second terminal as computer 208 (Figure 2A, 208), wherein the picture as the captured image (Chang, ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”) contains a target as the target of the image (Chang, ¶67 “In examples similar to those discussed above, the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”) file as the requested data may be a file (Chang, ¶57 “In some cases, an application can attach offline resources or files.”) [[]] currently displayed on the screen as the image is of the display of computer 204 (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”) of the first terminal as computer 204 (Id);
[b] Capturing (Chang, ¶41 “In particular, a computer 204 may take a screenshot of its own screen”), by the first terminal as computer 204 (Id), a screenshot as a screenshot (Id) of the first terminal as its own screen, e.g. the screen of computer 204 (Id), wherein the screenshot contains a plurality of file [[]] as selecting an application from multiple possible applications (Chang, ¶15 “analyzing the digital image can include using optical character recognition to identify words, and identifying a currently operating application by using the words to select an application from multiple possible applications.”; ¶16 “The system includes a first computing device arranged to present one or more applications, each having a current state, on a display screen,”) currently displayed on the screen as the image is of the display of computer 204 (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”; ¶45), wherein the plurality of file [[]] as the multiple possible applications (Chang, ¶15 , ¶16) correspond to a plurality of files (Chang, ¶57 “Each attached resource or file can be stored in a JSON structure with two pairs of key-values (e.g., (1) a name, which is the file name, and (2) a content, which is the BASE64-encoded content of the file). The attachments can be stored in a JSON array identified by the key "files" in the reply message.”) stored (Chang, ¶42 “the applications on computer 204 may be programmed by their developers to perform such actions using an application programming interface (API)”; ¶57) in the first terminal as computer 204 (Id);
[c] comparing as compare (Chang, ¶41 “a computer 204 may take a screenshot of its own screen and compare that screenshot to the image received from computer 208”), by the first terminal as computer 204 (Id), the picture as the image (Id; ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”)) containing the target as the target of the image (Chang, ¶67) file (Chang, ¶57) [[ as the screenshot (Chang, ¶41) containing the plurality of file [[ (Chang, ¶15);
[d] based on comparison results (Chang, ¶41 “a computer 204 may take a screenshot of its own screen and compare that screenshot to the image received from computer 208”), determining, by the first terminal as computer 204 (Id), a degree of similarity as determining if a region matches or not (Chang, ¶41 “by paring each key point in the screenshot with each key point of the picture. Once the computer 204 has identified a matched region on the screen”; ¶43 “compared it to a screenshot of their current displays, determined that they were not the target of the communication and stayed quiet, or returned respective messages to the computer 208 that indicated they did not have a match and would not be providing current state information for themselves”) between the picture as the image (Id; ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”)) containing the target as the target of the image (Chang, ¶67) file (Chang, ¶57) [[as matched region (Chang, ¶55 “When the dispatcher 234 has found a matched region in the screenshot”) in the screenshot as the screenshot (Id) reaching a [[ as determining that there is a match using the SURF comparison algorithm (Chang, ¶55), wherein a position of the matched area in the screenshot is obtained as the coordinates of the matched region (Chang, ¶41 “Once the computer 204 has identified a matched region on the screen, it may obtain the X, Y coordinates of the comers of the region and the central point of the region and provide that information to the front-most application running on the computer 204 in that region.”);
[e] identifying, based on the position of the matched area in the screenshot as the coordinates of the matched region (Chang, ¶41 “Once the computer 204 has identified a matched region on the screen, it may obtain the X, Y coordinates of the comers of the region and the central point of the region and provide that information to the front-most application running on the computer 204 in that region.”), by the first terminal as computer 204 (Figure 2A, 204), a matched file [[ (Chang, ¶67 “the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”; ¶69 “to thereby infer what application is in the image and the state of that application”; Please note this claim limitation has been understood to be identifying the --target file icon--. Please see claim interpretation section for more detail) from the plurality of file [[ (Chang, ¶15, ¶16, ¶57);
[f] in response to identifying the matched file [[ (Chang, ¶67), determining, automatically by the first terminal, a target file path as the URL (Chang, ¶59 “For example, a mapping application may not show the URL to the current region of the map on the address bar. To get the real URL that represents the current state (the current region of the map), the web dispatcher extension 236 can inject a content script that calls "document.getElementByid('link').href' to obtain the real URL stored in the document object module (DOM) element with the id "link". Using this content script provides a user the ability to use deep shooting to open a map displaying on her computer in one step.”) or the URI (Chang, ¶44 “The URI or other form of information that describes the state of the application on the computer 204 is passed back by way of a corresponding text message to the computer 208 (arrow 218).”) corresponding to the matched file [[ (Chang, ¶67);
[g] in response to the matched file [[ (Chang, ¶67 “the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”; ¶69 “to thereby infer what application is in the image and the state of that application”) in the matched area as matched region (Chang, ¶55 “When the dispatcher 234 has found a matched region in the screenshot”) being identified as the matching (Chang, ¶67) and the target file path being determined as the URL (Chang, ¶59) or URI (Chang, ¶44), selecting as computer 204 determining from the match the appropriate application or applications to launch (Chang, ¶45 “If the computer 204 determines from the matching that it is the target computer, it may launch or make active an appropriate application or applications, and may set the state of those applications according to the information it received in the text message from computer 208.”) and [[ as computer 204 (Chang, ¶42), the matched file [[ (Chang, ¶40 “device may automatically check whenever an image is captured to determine if the image shows a computer display, and if it does, the functionality discussed here may be instigated automatically.”);
[h] obtaining, by the first terminal, a target file corresponding to the matched file [[as the computer 204 setting the state of the application, e.g. opening to the specific file (Chang, Chang, ¶45 “If the computer 204 determines from the matching that it is the target computer, it may launch or make active an appropriate application or applications, and may set the state of those applications according to the information it received in the text message from computer 208.”; ¶57 “The application then determines its state in any relevant manner and returns its state information to the dispatcher (box 316)… In some cases, an application can attach offline resources or files.”); and
[i] transmitting, by the first terminal, the target file as the reply includes the file (e.g the identified state) as an attachment (Chang, ¶57 “he application then determines its state in any relevant manner and returns its state information to the dispatcher (box 316)… In some cases, an application can attach offline resources or files. Each attached resource or file can be stored in a JSON structure with two pairs of key-values (e.g., (1) a name, which is the file name, and (2) a content, which is the BASE64-encoded content of the file). The attachments can be stored in a JSON array identified by the key "files" in the reply message.”; Please see 103 modification below) to the second terminal as opening the document on computer 208 (Chang, ¶45 “In the example above, the computer 204 may launch a hosted word processing system and may open the same document that the user was editing on the computer 208, so that the user may continue the editing process easily and seamlessly.”; ¶86 “People can use particular software to synchronize a mobile device with a desktop device”; ¶87 “USB sticks (USB flash drives) can be used to share files among computers. People can save the information they need to share as files, copy the files onto the USB stick, and then take the USB stick to another computer. In this type of scenario, deep shooting can be used to extract information automatically from an application running on a computer (e.g., a desktop computer) to a mobile device. The mobile device can be taken to another place, and then the user can post the extracted information to another computer ( e.g., another desktop computer) with deep posting.”; ¶88 “can be automatically transferred”; ¶89 “send game status information”).
Chang does not explicitly teach determining… a degree of similarity… reaching a threshold. It should be noted that Chang does teach determining similarity of a matched area as detailed above. Chang recites using the SURF comparison algorithm, but does not explicitly state the degree of similarity reaching a threshold.
Huijuan teaches [d] a degree of similarity as the ratio of distance (Page 1462 “The matching algorithm: Pa, Pb is one of the interest points. DesA,DesB are the descriptors components of Pa, Pb. DesAi,DesBi are the i weight of DesA,DesB. The distance of Pa, Pb is defined as follows: [Equation 6]. And we find out the Nearest Distance, and Next Nearest Distance. The Ratio of Distance is as follows:[Equation 7]. We set the ratio as 0.65. If The Ratio of Distance < 0.65, the two interest points in two images match.”)… reaching a threshold as being less than 0.65 (Id).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have improved the SURF used by Chang with the techniques taught by Huijuan as it yields the predictable results attaining real-time matching speed (Page 1460 “In order to implement real-time match on computer, we choose the fast feature-based algorithm SURF [4]. However, SURF has not attained real-time performance, so we choose the FAST [5] corner detector to replace Fast-=Hessian detector to improve the SURF to attain real-time requirement”). The “base” device being the SURF algorithm used by Chang. The “comparable” device being the improved SURF taught by Huijuan. The predictable result being the real time speed. (KSR rational C: MPEP 2143(C)).
Chang does not explicitly teach a plurality of file icons. Chang teaches that there are multiple applications on the device ¶42 and teaches identifying the desired application as the ‘topmost application’ ¶67.
Amanieux teaches [a] the picture contains a target file as document 202 (Amanieux, ¶25 “FIG. 2 is a diagram illustrating monitor 101 displaying document 202 having marking indicia 204 configured according to one embodiment of the present teachings. Desktop 200 rendered on monitor 101 displays desktop items 201 and document 202 within window pane 203. Desktop items 201 are icons, such as for "Recycle Bin", "My Documents" and "My Computer," which are found commonly on a desktop of the WINDOWS® operating system. For purposes of FIG. 2, document 202 is a MICROSOFT WORD® document, available from Microsoft Corporation. The MICROSOFT WORD® application displays document 202 within its own window container, window pane 203. The operating system displays marking indicia 204 onto the title bar of window pane 203. Marking indicia 204 is a visual indicator that represents information about document 202, such as its storage address or location and information about the address or location of computer system 10 (FIG. 1).”) icon as the marking indicia, such as desktop icons (¶25 “Desktop items 201 are icons… Marking indicia 204 is a visual indicator that represents information about document 202, such as its storage address or location and information about the address or location of computer system 10”; Please see 103 modification below) currently displayed on the screen as monitor 101 displaying the document (Id) …
[b] …a plurality of file icons as the desktop items 201 such as the recycle bin, my documents, and my computer (Id)… wherein the plurality of file icons (Id)
[c]... the picture containing the target file as document 202 (¶25) icon as the marking indicia (¶25) with the screenshot containing the plurality of file icons as the desktop items 201 (¶25);
[d] ...the picture containing the target file as document 202 (¶25) icon as the marking indicia (¶25)...;
[e] identifying...a matched file icon as the marking indicia of the requested file (¶25) ...from the plurality of file icons as the desktop items 201 (¶25);
[f] in response to identifying the matched file icon as the marking indicia of the requested file (¶25)
[g] in response to the matched file icon… the matched file icon… as the marking indicia of the requested file (¶25)
[h] obtaining... a target file (¶31 “Specially programmed computer 40 copies document 406 and transmits it to the mobile device over the wireless connection”) corresponding (¶26 “It should be noted that marking indicia 204 may comprise any number of different types of visual indicators, such as textual information, graphical information, a bar code, encoded text, or the like. The information that marking indicia 204 represents is the location information of document 202 and computer system 10 (FIG. 1).”) to the matched file icon as the marking indicia of the requested file (¶25).
[i] transmitting, by the first terminal, the target file to the second terminal (¶29 “portable device 30 may retrieve a copy of the actual document instead of merely a binary copy of the document”)
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to operate on images containing additional desktop items as depicted by Amanieux as such items are common found on desktops (Amanieux, ¶25), and are likely to be found in the photo of the display screen. One of ordinary skill in the art would readably recognize the ability to use the improved photo analyzing technology taught by the proposed combination (Cheng, ¶41 modified in view of Hiujuan) to identify the marking indicia taught by Amanieux (Amanieux, ¶26) to facilitate identifying the top most application (Cheng, ¶67) and associating that with the document location (Amanieux, ¶26; Cheng, ¶42).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to transfer not only the Base64-encoded contents of the file (Cheng, ¶57) but instead an entire copy of the actual document (Amanieux, ¶29) as it yields the predictable results of ensuring that the document be transferred to the user as requested (Amanieux, ¶2; ¶4; Cheng, ¶4).
The combination of Cheng-Amanieux does not explicitly teach that the marking indicia includes icons.
Bhatia teaches [a] the target file icon (Bhatia ¶53 “An icon or a name can represent the one of the target file and the file that identifies the location of the target file within the user interface”); …[b] a plurality of file icons… wherein the plurality of file icons (Bhatia ¶53); [c] the target file icon (Bhatia ¶53)… the plurality of file icons(Bhatia ¶53); [d] … the target file icon (Bhatia ¶53); [e] … a matched file icon… from the plurality of file icons (Bhatia ¶53); [f] … the matched file icon… the matched file icon (Bhatia ¶53); [g] … file icon… file icon (Bhatia ¶53); [h] … file icon (Bhatia ¶53).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to include icons as marking indicia as such information may reasonably be used to identify the location of a file (Bhatia, ¶53). Please note within the proposed combination, the marking indicia (Amanieux, ¶26) may be any graphical information which represents the location information of the document. When read in light of Bhatia, one of ordinary skill in the art would recognize icons as satisfying the requirements of being such marching indicia, as they are graphical information, which may represent the location of the target file (Bhatia, ¶53).
One of ordinary skill in the art would recognize that the proposed combination, when taken as a whole, would enable the device to analyze the photograph (Chang, ¶41) to identify marking indicia (Amanieux, ¶25. ¶26) including icons (Bhatia, ¶53) that is present in both the photograph and the screenshot (Chang, ¶41). The proposed combination would then be able to use those icons (Bhatia, ¶53) to identify the location of the target file (Chang, ¶59; Bhatia, ¶53), which may be used to identify the requested information (Cheng, ¶41, ¶42) which may be used to identify the content (Cheng, ¶44) which may include files (Chang, ¶57; Amanieux, ¶29) to be directly transmitted (Chang, ¶57) via peer-to-peer transmission (Amanieux, ¶29, ¶31) or through a document exchange server (Amanieux, ¶38).
Cheng does not explicitly teach highlighting.
Stone teaches [g] … selecting and highlighting (Stone, Column 27, lines 33-35 “he viewing operation region may be used to point out, or highlight, an object in an image that matches both selection criteria for object selection and object characteristic criteria for furthering narrowing object selection”)… the matched as matches (Id) icon as the object in the image (Id).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to point out or highlight the object that matches the selection criteria as depicted by Stone as it yields the predictable results of providing information regarding the computers operation to the user, providing confirmation to the user of the selected display object (Stone, column 47, lines 63-65 "The viewing operation may optionally provide confirmation to the user of the selected display object, such as by highlighting the selected display object.").
With regard to claims 2, 10, and 18 the proposed combination further teaches wherein: the picture as the captured image (Chang, ¶39 “the example shown in FIG. 2A when computer 208 is used to capture an image of the display on computer 204.”) is photographed (Chang, ¶27 “In the examples discussed here, a camera on a computing device, initiating device or initial computer ( e.g., a smartphone ), can be used to capture an image of a display screen of another computer or target computer”) by the second terminal as computer 208, e.g. the cell phone with the camera on it (Chang, ¶39 “the example shown in FIG. 2A when computer 208 is used to capture an image of the display on computer 204.”), the photograph is of the screen as the display of computer 204 (Id) of the first terminal as computer 204 (Id), and the screen as image displayed on the monitor (Chang, ¶39 “In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204.”) of the first terminal as computer 204 (Id) displays as in the example the monitor displaying the photo of three people (Id) the target file as the photo (Id) icon as the Marking Indicia (Amanieux ¶25) specifically an icon (Bhatia ¶53).
With regard to claims 3 and 11 the proposed combination further teaches wherein: the target file icon as the Marking Indicia (Amanieux ¶25 “Marking indicia 204 is a visual indicator that represents information about document 202, such as its storage address or location and information about the address or location of computer system 10”) specifically being an icon (Bhatia ¶53 “An icon or a name can represent the one of the target file and the file that identifies the location of the target file within the user interface”) represents the target file (Id) from the first terminal as the computer device 10 (Amanieux, ¶25), e.g. computer 204 (Chang, Figure 2).
With regard to claims 7 and 15, the propose combination further teaches wherein the receiving the picture from the second terminal comprises:
receiving the picture from a server establishing a connection via a document exchange server (Amanieux, ¶38 “FIG. 7 by document exchange server 708, portable device 705 establishes a wireless connection, connection 710, with the wireless-enabled electronic device locally connected to computer system 700.”), wherein the picture is photographed (Chang, ¶27 “In the examples discussed here, a camera on a computing device, initiating device or initial computer ( e.g., a smartphone ), can be used to capture an image of a display screen of another computer or target computer”; Amanieux, ¶24 “Mobile phone 105 includes camera functionality with lens 106 along with a document exchange application stored in camera memory (not shown).”) and uploaded to the server by the second terminal (Amanieux, ¶38 “As such, any copy requests or paste requests will be passed through the locally connected wireless device.”; Figure 7, information passes from mobile device 705 to exchange server 708 to computer system 700).
With regard to claims 8 and 16 the proposed combination further teaches wherein the receiving the picture from the second terminal comprises:
directly receiving the picture from the second terminal as establishing a peer-to-peer connection (Amanieux, ¶31 “Using an available wireless protocol, the mobile device establishes a peer-to-peer communication link with specially programmed computer 40.”).
With regard to claim 9 Chang teaches A terminal as the Computer 204 (Chang, Figure 2A, 204; Figure 3A-B “Target Computer”; ¶32 “Three main components of the system 200 include a desktop computer 204, a computer 208 (which may be a mobile computer), and a server system 206.”) for transmitting (Chang, ¶5 “mechanisms for moving information conveniently between computing devices”) a target as the target of the image (Chang, ¶67 “In examples similar to those discussed above, the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”) file as the requested data may be a file (Chang, ¶57 “In some cases, an application can attach offline resources or files.”) to a second terminal as computer 208 (Figure 2A, 208), the terminal comprising:
a memory as memory 704 (Chang, ¶91 “The processor 702 can process instructions for execution within the computing device 700, including instructions stored in the memory 704”) storing instructions as instructions (Id); and
a processor as processor 702 (Id) in communication with the memory (Id), wherein, when the processor executes the instructions, the processor is configured to cause the terminal to:
[a] Receive (Chang, ¶41 “the computer 204 receives the message from computer 208”) a picture as the message contains the captured image (Chang, ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”) comprising at least a portion as at least a portion of the image (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”) of a screen as the image displayed on the monitory of computer 204 (Id) of the terminal as computer 204 (Id) from as computer 208 captures the image and sends the message (Chang, ¶39 and ¶41) the second terminal as computer 208 (Figure 2A, 208), wherein the picture as the captured image (Chang, ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”) contains a target as the target of the image (Chang, ¶67 “In examples similar to those discussed above, the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”) file as the requested data may be a file (Chang, ¶57 “In some cases, an application can attach offline resources or files.”) [[]] currently displayed on the screen as the image is of the display of computer 204 (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”) of the terminal as computer 204 (Id);
[b] Capture (Chang, ¶41 “In particular, a computer 204 may take a screenshot of its own screen”) a screenshot as a screenshot (Id) of the terminal as its own screen, e.g. the screen of computer 204 (Id), wherein the screenshot contains a plurality of file [[]] as selecting an application from multiple possible applications (Chang, ¶15 “analyzing the digital image can include using optical character recognition to identify words, and identifying a currently operating application by using the words to select an application from multiple possible applications.”; ¶16 “The system includes a first computing device arranged to present one or more applications, each having a current state, on a display screen,”) currently displayed on the screen as the image is of the display of computer 204 (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”; ¶45), wherein the plurality of file [[]] as the multiple possible applications (Chang, ¶15 , ¶16) correspond to a plurality of files (Chang, ¶57 “Each attached resource or file can be stored in a JSON structure with two pairs of key-values (e.g., (1) a name, which is the file name, and (2) a content, which is the BASE64-encoded content of the file). The attachments can be stored in a JSON array identified by the key "files" in the reply message.”) stored (Chang, ¶42 “the applications on computer 204 may be programmed by their developers to perform such actions using an application programming interface (API)”; ¶57) in the terminal as computer 204 (Id);
[c] compare as compare (Chang, ¶41 “a computer 204 may take a screenshot of its own screen and compare that screenshot to the image received from computer 208”) the picture as the image (Id; ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”)) containing the target as the target of the image (Chang, ¶67) file (Chang, ¶57) [[ as the screenshot (Chang, ¶41) containing the plurality of file [[ (Chang, ¶15);
[d] based on comparison results (Chang, ¶41 “a computer 204 may take a screenshot of its own screen and compare that screenshot to the image received from computer 208”), determine a degree of similarity as determining if a region matches or not (Chang, ¶41 “by paring each key point in the screenshot with each key point of the picture. Once the computer 204 has identified a matched region on the screen”; ¶43 “compared it to a screenshot of their current displays, determined that they were not the target of the communication and stayed quiet, or returned respective messages to the computer 208 that indicated they did not have a match and would not be providing current state information for themselves”) between the picture as the image (Id; ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”)) containing the target as the target of the image (Chang, ¶67) file (Chang, ¶57) [[matched area as matched region (Chang, ¶55 “When the dispatcher 234 has found a matched region in the screenshot”) in the screenshot as the screenshot (Id) reaching a [[ as determining that there is a match using the SURF comparison algorithm (Chang, ¶55), wherein a position of the matched area in the screenshot is obtained as the coordinates of the matched region (Chang, ¶41 “Once the computer 204 has identified a matched region on the screen, it may obtain the X, Y coordinates of the comers of the region and the central point of the region and provide that information to the front-most application running on the computer 204 in that region.”);
[e] identify, based on the position of the matched area in the screenshot as the coordinates of the matched region (Chang, ¶41 “Once the computer 204 has identified a matched region on the screen, it may obtain the X, Y coordinates of the comers of the region and the central point of the region and provide that information to the front-most application running on the computer 204 in that region.”), by the first terminal as computer 204 (Figure 2A, 204), a matched file [[ (Chang, ¶67 “the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”; ¶69 “to thereby infer what application is in the image and the state of that application”; Please note this claim limitation has been understood to be identifying the --target file icon--. Please see claim interpretation section for more detail) from the plurality of file [[ (Chang, ¶15, ¶16, ¶57);
[f] in response to identifying the matched file [[ (Chang, ¶67), determine, automatically by the terminal, a target file path as the URL (Chang, ¶59 “For example, a mapping application may not show the URL to the current region of the map on the address bar. To get the real URL that represents the current state (the current region of the map), the web dispatcher extension 236 can inject a content script that calls "document.getElementByid('link').href' to obtain the real URL stored in the document object module (DOM) element with the id "link". Using this content script provides a user the ability to use deep shooting to open a map displaying on her computer in one step.”) or the URI (Chang, ¶44 “The URI or other form of information that describes the state of the application on the computer 204 is passed back by way of a corresponding text message to the computer 208 (arrow 218).”) corresponding to the matched file [[ (Chang, ¶67);
[g] in response to the matched file [[ (Chang, ¶67 “the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”; ¶69 “to thereby infer what application is in the image and the state of that application”) in the matched area as matched region (Chang, ¶55 “When the dispatcher 234 has found a matched region in the screenshot”) being identified as the matching (Chang, ¶67) and the target file path being determined as the URL (Chang, ¶59) or URI (Chang, ¶44), select as computer 204 determining from the match the appropriate application or applications to launch (Chang, ¶45 “If the computer 204 determines from the matching that it is the target computer, it may launch or make active an appropriate application or applications, and may set the state of those applications according to the information it received in the text message from computer 208.”) and [[ as computer 204 (Chang, ¶42), the matched file [[ (Chang, ¶40 “device may automatically check whenever an image is captured to determine if the image shows a computer display, and if it does, the functionality discussed here may be instigated automatically.”);
[h] obtain a target file corresponding to the matched file [[as the computer 204 setting the state of the application, e.g. opening to the specific file (Chang, Chang, ¶45 “If the computer 204 determines from the matching that it is the target computer, it may launch or make active an appropriate application or applications, and may set the state of those applications according to the information it received in the text message from computer 208.”; ¶57 “The application then determines its state in any relevant manner and returns its state information to the dispatcher (box 316)… In some cases, an application can attach offline resources or files.”); and
[i] transmit the target file as the reply includes the file (e.g the identified state) as an attachment (Chang, ¶57 “he application then determines its state in any relevant manner and returns its state information to the dispatcher (box 316)… In some cases, an application can attach offline resources or files. Each attached resource or file can be stored in a JSON structure with two pairs of key-values (e.g., (1) a name, which is the file name, and (2) a content, which is the BASE64-encoded content of the file). The attachments can be stored in a JSON array identified by the key "files" in the reply message.”; Please see 103 modification below) to the second terminal as opening the document on computer 208 (Chang, ¶45 “In the example above, the computer 204 may launch a hosted word processing system and may open the same document that the user was editing on the computer 208, so that the user may continue the editing process easily and seamlessly.”; ¶86 “People can use particular software to synchronize a mobile device with a desktop device”; ¶87 “USB sticks (USB flash drives) can be used to share files among computers. People can save the information they need to share as files, copy the files onto the USB stick, and then take the USB stick to another computer. In this type of scenario, deep shooting can be used to extract information automatically from an application running on a computer (e.g., a desktop computer) to a mobile device. The mobile device can be taken to another place, and then the user can post the extracted information to another computer ( e.g., another desktop computer) with deep posting.”; ¶88 “can be automatically transferred”; ¶89 “send game status information”).
Chang does not explicitly teach determining… a degree of similarity… reaching a threshold. It should be noted that Chang does teach determining similarity of a matched area as detailed above. Chang recites using the SURF comparison algorithm, but does not explicitly state the degree of similarity reaching a threshold.
Huijuan teaches [d] a degree of similarity as the ratio of distance (Page 1462 “The matching algorithm: Pa, Pb is one of the interest points. DesA,DesB are the descriptors components of Pa, Pb. DesAi,DesBi are the i weight of DesA,DesB. The distance of Pa, Pb is defined as follows: [Equation 6]. And we find out the Nearest Distance, and Next Nearest Distance. The Ratio of Distance is as follows:[Equation 7]. We set the ratio as 0.65. If The Ratio of Distance < 0.65, the two interest points in two images match.”)… reaching a threshold as being less than 0.65 (Id).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have improved the SURF used by Chang with the techniques taught by Huijuan as it yields the predictable results attaining real-time matching speed (Page 1460 “In order to implement real-time match on computer, we choose the fast feature-based algorithm SURF [4]. However, SURF has not attained real-time performance, so we choose the FAST [5] corner detector to replace Fast-=Hessian detector to improve the SURF to attain real-time requirement”). The “base” device being the SURF algorithm used by Chang. The “comparable” device being the improved SURF taught by Huijuan. The predictable result being the real time speed. (KSR rational C: MPEP 2143(C)).
Chang does not explicitly teach a plurality of file icons. Chang teaches that there are multiple applications on the device ¶42 and teaches identifying the desired application as the ‘topmost application’ ¶67.
Amanieux teaches [a] the picture contains a target file as document 202 (Amanieux, ¶25 “FIG. 2 is a diagram illustrating monitor 101 displaying document 202 having marking indicia 204 configured according to one embodiment of the present teachings. Desktop 200 rendered on monitor 101 displays desktop items 201 and document 202 within window pane 203. Desktop items 201 are icons, such as for "Recycle Bin", "My Documents" and "My Computer," which are found commonly on a desktop of the WINDOWS® operating system. For purposes of FIG. 2, document 202 is a MICROSOFT WORD® document, available from Microsoft Corporation. The MICROSOFT WORD® application displays document 202 within its own window container, window pane 203. The operating system displays marking indicia 204 onto the title bar of window pane 203. Marking indicia 204 is a visual indicator that represents information about document 202, such as its storage address or location and information about the address or location of computer system 10 (FIG. 1).”) icon as the marking indicia, such as desktop icons (¶25 “Desktop items 201 are icons… Marking indicia 204 is a visual indicator that represents information about document 202, such as its storage address or location and information about the address or location of computer system 10”; Please see 103 modification below) currently displayed on the screen as monitor 101 displaying the document (Id) …
[b] …a plurality of file icons as the desktop items 201 such as the recycle bin, my documents, and my computer (Id)… wherein the plurality of file icons (Id)
[c]... the picture containing the target file as document 202 (¶25) icon as the marking indicia (¶25) with the screenshot containing the plurality of file icons as the desktop items 201 (¶25);
[d] ...the picture containing the target file as document 202 (¶25) icon as the marking indicia (¶25)...;
[e] identify...a matched file icon as the marking indicia of the requested file (¶25) ...from the plurality of file icons as the desktop items 201 (¶25);
[f] in response to identifying the matched file icon as the marking indicia of the requested file (¶25)
[g] in response to the matched file icon… the matched file icon… as the marking indicia of the requested file (¶25)
[h] obtain... a target file (¶31 “Specially programmed computer 40 copies document 406 and transmits it to the mobile device over the wireless connection”) corresponding (¶26 “It should be noted that marking indicia 204 may comprise any number of different types of visual indicators, such as textual information, graphical information, a bar code, encoded text, or the like. The information that marking indicia 204 represents is the location information of document 202 and computer system 10 (FIG. 1).”) to the matched file icon as the marking indicia of the requested file (¶25).
[i] transmit the target file to the second terminal (¶29 “portable device 30 may retrieve a copy of the actual document instead of merely a binary copy of the document”)
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to operate on images containing additional desktop items as depicted by Amanieux as such items are common found on desktops (Amanieux, ¶25), and are likely to be found in the photo of the display screen. One of ordinary skill in the art would readably recognize the ability to use the improved photo analyzing technology taught by the proposed combination (Cheng, ¶41 modified in view of Hiujuan) to identify the marking indicia taught by Amanieux (Amanieux, ¶26) to facilitate identifying the top most application (Cheng, ¶67) and associating that with the document location (Amanieux, ¶26; Cheng, ¶42).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to transfer not only the Base64-encoded contents of the file (Cheng, ¶57) but instead an entire copy of the actual document (Amanieux, ¶29) as it yields the predictable results of ensuring that the document be transferred to the user as requested (Amanieux, ¶2; ¶4; Cheng, ¶4).
The combination of Cheng-Amanieux does not explicitly teach that the marking indicia includes icons.
Bhatia teaches [a] the target file icon (Bhatia ¶53 “An icon or a name can represent the one of the target file and the file that identifies the location of the target file within the user interface”); …[b] a plurality of file icons… wherein the plurality of file icons (Bhatia ¶53); [c] the target file icon (Bhatia ¶53)… the plurality of file icons(Bhatia ¶53); [d] … the target file icon (Bhatia ¶53); [e] … a matched file icon… from the plurality of file icons (Bhatia ¶53); [f] … the matched file icon… the matched file icon (Bhatia ¶53); [g] … file icon… file icon (Bhatia ¶53); [h] … file icon (Bhatia ¶53).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to include icons as marking indicia as such information may reasonably be used to identify the location of a file (Bhatia, ¶53). Please note within the proposed combination, the marking indicia (Amanieux, ¶26) may be any graphical information which represents the location information of the document. When read in light of Bhatia, one of ordinary skill in the art would recognize icons as satisfying the requirements of being such marching indicia, as they are graphical information, which may represent the location of the target file (Bhatia, ¶53).
One of ordinary skill in the art would recognize that the proposed combination, when taken as a whole, would enable the device to analyze the photograph (Chang, ¶41) to identify marking indicia (Amanieux, ¶25. ¶26) including icons (Bhatia, ¶53) that is present in both the photograph and the screenshot (Chang, ¶41). The proposed combination would then be able to use those icons (Bhatia, ¶53) to identify the location of the target file (Chang, ¶59; Bhatia, ¶53), which may be used to identify the requested information (Cheng, ¶41, ¶42) which may be used to identify the content (Cheng, ¶44) which may include files (Chang, ¶57; Amanieux, ¶29) to be directly transmitted (Chang, ¶57) via peer-to-peer transmission (Amanieux, ¶29, ¶31) or through a document exchange server (Amanieux, ¶38).
Cheng does not explicitly teach highlight.
Stone teaches [g] … select and highlight (Stone, Column 27, lines 33-35 “he viewing operation region may be used to point out, or highlight, an object in an image that matches both selection criteria for object selection and object characteristic criteria for furthering narrowing object selection”)… the matched as matches (Id) icon as the object in the image (Id).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to point out or highlight the object that matches the selection criteria as depicted by Stone as it yields the predictable results of providing information regarding the computers operation to the user, providing confirmation to the user of the selected display object (Stone, column 47, lines 63-65 "The viewing operation may optionally provide confirmation to the user of the selected display object, such as by highlighting the selected display object.").
With regard to claim 17 Chang teaches A non-transitory computer-readable storage medium as memory 704 (Chang, ¶91 “The processor 702 can process instructions for execution within the computing device 700, including instructions stored in the memory 704”) , storing computer-readable instructions as instructions (Id), wherein, the computer-readable instructions, when executed by a processor as processor 702 (Id) in a first terminal as the Computer 204 (Chang, Figure 2A, 204; Figure 3A-B “Target Computer”; ¶32 “Three main components of the system 200 include a desktop computer 204, a computer 208 (which may be a mobile computer), and a server system 206.”), are configured to cause the processor to perform:
[a] Receiving (Chang, ¶41 “the computer 204 receives the message from computer 208”) a picture as the message contains the captured image (Chang, ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”) comprising at least a portion as at least a portion of the image (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”) of a screen as the image displayed on the monitory of computer 204 (Id) of the first terminal as computer 204 (Id) from as computer 208 captures the image and sends the message (Chang, ¶39 and ¶41) a second terminal as computer 208 (Figure 2A, 208), wherein the picture as the captured image (Chang, ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”) contains a target as the target of the image (Chang, ¶67 “In examples similar to those discussed above, the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”) file as the requested data may be a file (Chang, ¶57 “In some cases, an application can attach offline resources or files.”) [[]] currently displayed on the screen as the image is of the display of computer 204 (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”) of the first terminal as computer 204 (Id);
[b] Capturing (Chang, ¶41 “In particular, a computer 204 may take a screenshot of its own screen”) a screenshot as a screenshot (Id) of the first terminal as its own screen, e.g. the screen of computer 204 (Id), wherein the screenshot contains a plurality of file [[]] as selecting an application from multiple possible applications (Chang, ¶15 “analyzing the digital image can include using optical character recognition to identify words, and identifying a currently operating application by using the words to select an application from multiple possible applications.”; ¶16 “The system includes a first computing device arranged to present one or more applications, each having a current state, on a display screen,”) currently displayed on the screen as the image is of the display of computer 204 (Chang, ¶39 “computer 208 is used to capture an image of the display on computer 204. In this example, a photo of three people can be displayed in a photo management application, including a hosted application, on the monitor of computer 204. The computer 208 is shown as having captured at least a portion of that image”; ¶45), wherein the plurality of file [[]] as the multiple possible applications (Chang, ¶15 , ¶16) correspond to a plurality of files (Chang, ¶57 “Each attached resource or file can be stored in a JSON structure with two pairs of key-values (e.g., (1) a name, which is the file name, and (2) a content, which is the BASE64-encoded content of the file). The attachments can be stored in a JSON array identified by the key "files" in the reply message.”) stored (Chang, ¶42 “the applications on computer 204 may be programmed by their developers to perform such actions using an application programming interface (API)”; ¶57) in the first terminal as computer 204 (Id);
[c] comparing as compare (Chang, ¶41 “a computer 204 may take a screenshot of its own screen and compare that screenshot to the image received from computer 208”) the picture as the image (Id; ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”)) containing the target as the target of the image (Chang, ¶67) file (Chang, ¶57) [[ as the screenshot (Chang, ¶41) containing the plurality of file [[ (Chang, ¶15);
[d] based on comparison results (Chang, ¶41 “a computer 204 may take a screenshot of its own screen and compare that screenshot to the image received from computer 208”), determining a degree of similarity as determining if a region matches or not (Chang, ¶41 “by paring each key point in the screenshot with each key point of the picture. Once the computer 204 has identified a matched region on the screen”; ¶43 “compared it to a screenshot of their current displays, determined that they were not the target of the communication and stayed quiet, or returned respective messages to the computer 208 that indicated they did not have a match and would not be providing current state information for themselves”) between the picture as the image (Id; ¶39 “The computer 208 is shown as having captured at least a portion of that image, passing the image along with other relevant information for the information sharing process to the text message server system 206 (arrow 214)”)) containing the target as the target of the image (Chang, ¶67) file (Chang, ¶57) [[as matched region (Chang, ¶55 “When the dispatcher 234 has found a matched region in the screenshot”) in the screenshot as the screenshot (Id) reaching a [[ as determining that there is a match using the SURF comparison algorithm (Chang, ¶55), wherein a position of the matched area in the screenshot is obtained as the coordinates of the matched region (Chang, ¶41 “Once the computer 204 has identified a matched region on the screen, it may obtain the X, Y coordinates of the comers of the region and the central point of the region and provide that information to the front-most application running on the computer 204 in that region.”);
[e] identifying, based on the position of the matched area in the screenshot as the coordinates of the matched region (Chang, ¶41 “Once the computer 204 has identified a matched region on the screen, it may obtain the X, Y coordinates of the comers of the region and the central point of the region and provide that information to the front-most application running on the computer 204 in that region.”), by the first terminal as computer 204 (Figure 2A, 204), a matched file [[ (Chang, ¶67 “the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”; ¶69 “to thereby infer what application is in the image and the state of that application”; Please note this claim limitation has been understood to be identifying the --target file icon--. Please see claim interpretation section for more detail) from the plurality of file [[ (Chang, ¶15, ¶16, ¶57);
[f] in response to identifying the matched file [[ (Chang, ¶67), determining, automatically by the first terminal, a target file path as the URL (Chang, ¶59 “For example, a mapping application may not show the URL to the current region of the map on the address bar. To get the real URL that represents the current state (the current region of the map), the web dispatcher extension 236 can inject a content script that calls "document.getElementByid('link').href' to obtain the real URL stored in the document object module (DOM) element with the id "link". Using this content script provides a user the ability to use deep shooting to open a map displaying on her computer in one step.”) or the URI (Chang, ¶44 “The URI or other form of information that describes the state of the application on the computer 204 is passed back by way of a corresponding text message to the computer 208 (arrow 218).”) corresponding to the matched file [[ (Chang, ¶67);
[g] in response to the matched file [[ (Chang, ¶67 “the aspects may include portions of the image that match an image taken of a screenshot for the computer that was the target of the first image.”; ¶69 “to thereby infer what application is in the image and the state of that application”) in the matched area as matched region (Chang, ¶55 “When the dispatcher 234 has found a matched region in the screenshot”) being identified as the matching (Chang, ¶67) and the target file path being determined as the URL (Chang, ¶59) or URI (Chang, ¶44), selecting as computer 204 determining from the match the appropriate application or applications to launch (Chang, ¶45 “If the computer 204 determines from the matching that it is the target computer, it may launch or make active an appropriate application or applications, and may set the state of those applications according to the information it received in the text message from computer 208.”) and [[ as computer 204 (Chang, ¶42), the matched file [[ (Chang, ¶40 “device may automatically check whenever an image is captured to determine if the image shows a computer display, and if it does, the functionality discussed here may be instigated automatically.”);
[h] obtaining a target file corresponding to the matched file [[as the computer 204 setting the state of the application, e.g. opening to the specific file (Chang, Chang, ¶45 “If the computer 204 determines from the matching that it is the target computer, it may launch or make active an appropriate application or applications, and may set the state of those applications according to the information it received in the text message from computer 208.”; ¶57 “The application then determines its state in any relevant manner and returns its state information to the dispatcher (box 316)… In some cases, an application can attach offline resources or files.”); and
[i] transmitting the target file as the reply includes the file (e.g the identified state) as an attachment (Chang, ¶57 “he application then determines its state in any relevant manner and returns its state information to the dispatcher (box 316)… In some cases, an application can attach offline resources or files. Each attached resource or file can be stored in a JSON structure with two pairs of key-values (e.g., (1) a name, which is the file name, and (2) a content, which is the BASE64-encoded content of the file). The attachments can be stored in a JSON array identified by the key "files" in the reply message.”; Please see 103 modification below) to the second terminal as opening the document on computer 208 (Chang, ¶45 “In the example above, the computer 204 may launch a hosted word processing system and may open the same document that the user was editing on the computer 208, so that the user may continue the editing process easily and seamlessly.”; ¶86 “People can use particular software to synchronize a mobile device with a desktop device”; ¶87 “USB sticks (USB flash drives) can be used to share files among computers. People can save the information they need to share as files, copy the files onto the USB stick, and then take the USB stick to another computer. In this type of scenario, deep shooting can be used to extract information automatically from an application running on a computer (e.g., a desktop computer) to a mobile device. The mobile device can be taken to another place, and then the user can post the extracted information to another computer ( e.g., another desktop computer) with deep posting.”; ¶88 “can be automatically transferred”; ¶89 “send game status information”).
Chang does not explicitly teach determining… a degree of similarity… reaching a threshold. It should be noted that Chang does teach determining similarity of a matched area as detailed above. Chang recites using the SURF comparison algorithm, but does not explicitly state the degree of similarity reaching a threshold.
Huijuan teaches [d] a degree of similarity as the ratio of distance (Page 1462 “The matching algorithm: Pa, Pb is one of the interest points. DesA,DesB are the descriptors components of Pa, Pb. DesAi,DesBi are the i weight of DesA,DesB. The distance of Pa, Pb is defined as follows: [Equation 6]. And we find out the Nearest Distance, and Next Nearest Distance. The Ratio of Distance is as follows:[Equation 7]. We set the ratio as 0.65. If The Ratio of Distance < 0.65, the two interest points in two images match.”)… reaching a threshold as being less than 0.65 (Id).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have improved the SURF used by Chang with the techniques taught by Huijuan as it yields the predictable results attaining real-time matching speed (Page 1460 “In order to implement real-time match on computer, we choose the fast feature-based algorithm SURF [4]. However, SURF has not attained real-time performance, so we choose the FAST [5] corner detector to replace Fast-=Hessian detector to improve the SURF to attain real-time requirement”). The “base” device being the SURF algorithm used by Chang. The “comparable” device being the improved SURF taught by Huijuan. The predictable result being the real time speed. (KSR rational C: MPEP 2143(C)).
Chang does not explicitly teach a plurality of file icons. Chang teaches that there are multiple applications on the device ¶42 and teaches identifying the desired application as the ‘topmost application’ ¶67.
Amanieux teaches [a] the picture contains a target file as document 202 (Amanieux, ¶25 “FIG. 2 is a diagram illustrating monitor 101 displaying document 202 having marking indicia 204 configured according to one embodiment of the present teachings. Desktop 200 rendered on monitor 101 displays desktop items 201 and document 202 within window pane 203. Desktop items 201 are icons, such as for "Recycle Bin", "My Documents" and "My Computer," which are found commonly on a desktop of the WINDOWS® operating system. For purposes of FIG. 2, document 202 is a MICROSOFT WORD® document, available from Microsoft Corporation. The MICROSOFT WORD® application displays document 202 within its own window container, window pane 203. The operating system displays marking indicia 204 onto the title bar of window pane 203. Marking indicia 204 is a visual indicator that represents information about document 202, such as its storage address or location and information about the address or location of computer system 10 (FIG. 1).”) icon as the marking indicia, such as desktop icons (¶25 “Desktop items 201 are icons… Marking indicia 204 is a visual indicator that represents information about document 202, such as its storage address or location and information about the address or location of computer system 10”; Please see 103 modification below) currently displayed on the screen as monitor 101 displaying the document (Id) …
[b] …a plurality of file icons as the desktop items 201 such as the recycle bin, my documents, and my computer (Id)… wherein the plurality of file icons (Id)
[c]... the picture containing the target file as document 202 (¶25) icon as the marking indicia (¶25) with the screenshot containing the plurality of file icons as the desktop items 201 (¶25);
[d] ...the picture containing the target file as document 202 (¶25) icon as the marking indicia (¶25)...;
[e] identifying...a matched file icon as the marking indicia of the requested file (¶25) ...from the plurality of file icons as the desktop items 201 (¶25);
[f] in response to identifying the matched file icon as the marking indicia of the requested file (¶25)
[g] in response to the matched file icon… the matched file icon… as the marking indicia of the requested file (¶25)
[h] obtaining... a target file (¶31 “Specially programmed computer 40 copies document 406 and transmits it to the mobile device over the wireless connection”) corresponding (¶26 “It should be noted that marking indicia 204 may comprise any number of different types of visual indicators, such as textual information, graphical information, a bar code, encoded text, or the like. The information that marking indicia 204 represents is the location information of document 202 and computer system 10 (FIG. 1).”) to the matched file icon as the marking indicia of the requested file (¶25).
[i] transmitting the target file to the second terminal (¶29 “portable device 30 may retrieve a copy of the actual document instead of merely a binary copy of the document”)
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to operate on images containing additional desktop items as depicted by Amanieux as such items are common found on desktops (Amanieux, ¶25), and are likely to be found in the photo of the display screen. One of ordinary skill in the art would readably recognize the ability to use the improved photo analyzing technology taught by the proposed combination (Cheng, ¶41 modified in view of Hiujuan) to identify the marking indicia taught by Amanieux (Amanieux, ¶26) to facilitate identifying the top most application (Cheng, ¶67) and associating that with the document location (Amanieux, ¶26; Cheng, ¶42).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to transfer not only the Base64-encoded contents of the file (Cheng, ¶57) but instead an entire copy of the actual document (Amanieux, ¶29) as it yields the predictable results of ensuring that the document be transferred to the user as requested (Amanieux, ¶2; ¶4; Cheng, ¶4).
The combination of Cheng-Amanieux does not explicitly teach that the marking indicia includes icons.
Bhatia teaches [a] the target file icon (Bhatia ¶53 “An icon or a name can represent the one of the target file and the file that identifies the location of the target file within the user interface”); …[b] a plurality of file icons… wherein the plurality of file icons (Bhatia ¶53); [c] the target file icon (Bhatia ¶53)… the plurality of file icons(Bhatia ¶53); [d] … the target file icon (Bhatia ¶53); [e] … a matched file icon… from the plurality of file icons (Bhatia ¶53); [f] … the matched file icon… the matched file icon (Bhatia ¶53); [g] … file icon… file icon (Bhatia ¶53); [h] … file icon (Bhatia ¶53).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to include icons as marking indicia as such information may reasonably be used to identify the location of a file (Bhatia, ¶53). Please note within the proposed combination, the marking indicia (Amanieux, ¶26) may be any graphical information which represents the location information of the document. When read in light of Bhatia, one of ordinary skill in the art would recognize icons as satisfying the requirements of being such marching indicia, as they are graphical information, which may represent the location of the target file (Bhatia, ¶53).
One of ordinary skill in the art would recognize that the proposed combination, when taken as a whole, would enable the device to analyze the photograph (Chang, ¶41) to identify marking indicia (Amanieux, ¶25. ¶26) including icons (Bhatia, ¶53) that is present in both the photograph and the screenshot (Chang, ¶41). The proposed combination would then be able to use those icons (Bhatia, ¶53) to identify the location of the target file (Chang, ¶59; Bhatia, ¶53), which may be used to identify the requested information (Cheng, ¶41, ¶42) which may be used to identify the content (Cheng, ¶44) which may include files (Chang, ¶57; Amanieux, ¶29) to be directly transmitted (Chang, ¶57) via peer-to-peer transmission (Amanieux, ¶29, ¶31) or through a document exchange server (Amanieux, ¶38).
Cheng does not explicitly teach highlighting.
Stone teaches [g] … selecting and highlighting (Stone, Column 27, lines 33-35 “he viewing operation region may be used to point out, or highlight, an object in an image that matches both selection criteria for object selection and object characteristic criteria for furthering narrowing object selection”)… the matched as matches (Id) icon as the object in the image (Id).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the proposed combination to point out or highlight the object that matches the selection criteria as depicted by Stone as it yields the predictable results of providing information regarding the computers operation to the user, providing confirmation to the user of the selected display object (Stone, column 47, lines 63-65 "The viewing operation may optionally provide confirmation to the user of the selected display object, such as by highlighting the selected display object.").
Claims 4, 5, 12, 13, 19 and 20 are rejected under pre-AIA 35 U.S.C. 103(a) as being unpatentable over Chang in view of Huijuan, Amanieux, Bhatia, Stone, and StackOverflow [Is file path a url?].
With regard to claims 4, 12, and 19 the proposed combination further teaches wherein the obtaining the target file corresponding to the matched file icon comprises:
determining a target file name (Chang, ¶57 “file name”) of the target file according to a position as URL (Chang, ¶59) or the URI (Chang, ¶44) of the matched file as the topmost application (Id) icon as the application identified by the icon (Bhaita, ¶53) in [[
obtaining the target file (Amanieux, ¶31 “Specially programmed computer 40 copies document 406 and transmits it to the mobile device over the wireless connection”) corresponding as transmitting the message to the mobile device (Chang, ¶60) to the target file name (Chang, ¶57 “file name”).
Cheng does not explicitly teach the location of the file being in a file folder.
StackOverflow teaches in a file folder as the folder “files” within the address file://c:\files\someFile.dat” (See Post by Welbog on Page 1).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have recognized that the URI/URLs taught by the proposed combination (Chang, ¶44, ¶59) may specify file folders as demonstrated by StackOverflow. The proposed combination is combining known prior art elements (known structures of URL/URI) to the specifically recite URL/URIs which the proposed combination already teaches (KSR Rational A). This would yield the predictable results of the URL/URI identifying the location of the file, as is necessary to be able to transmit it.
With regard to claims 5, 13, and 20 the proposed combination further teaches determining a target file folder name of the target file according to the file folder as the folder “files” within the address file://c:\files\someFile.dat” (See Post by Welbog on Page 1) containing the target file icon as the Marking Indicia (Amanieux ¶25 “Marking indicia 204 is a visual indicator that represents information about document 202, such as its storage address or location and information about the address or location of computer system 10”) specifically being an icon (Bhatia ¶53 “An icon or a name can represent the one of the target file and the file that identifies the location of the target file within the user interface”);
constructing the target file path as URL (Chang, ¶59) or the URI (Chang, ¶44) based on the target file name (Chang, ¶57 “file name”) and the target file folder name as the folder “files” for the file “someFile.dag” within the address file://c:\files\someFile.dat” (See Post by Welbog on Page 1); and
wherein the obtaining the target file corresponding to the target file name comprises:
obtaining the target file (Amanieux, ¶31 “Specially programmed computer 40 copies document 406 and transmits it to the mobile device over the wireless connection”) corresponding as transmitting the message to the mobile device (Chang, ¶60) to the target file path as URL (Chang, ¶59) or the URI (Chang, ¶44).
Response to Arguments
Applicant's arguments filed February 24, 2025 have been fully considered but they are not persuasive. All the arguments regarding the newly added limitations are addressed in the above rejections.
With regard to the highlighting, applicant argues Chang does not teach the highlighting before making active an application.
In response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., before) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). To be clear, the claim language does not place any restriction on when the highlighting is done, instead it merely recites “selecting and highlighting”. There is no requirement in the claims for the highlighting be done before making the application active.
Furthermore, within the proposed combination, Stone teaches that the highlighting is done to enable the user to further narrow doesn’t the selection, which implies that the highlighting is done before the selected object is to be displayed to the user. The purpose of Stone’s highlighting is to enable the user to confirm that the user wants to display the selected object.
All other arguments are addressed in the above claim mapping.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
Please note that the Stone Reference (e.g. Pat [5467411]) was originally cited in the Notice of References Cited dated April 1, 2025.
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMANDA WILLIS whose telephone number is (571)270-7691. The examiner can normally be reached Monday-Friday 8am-2pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ajay Bhatia can be reached at 571-272-3906. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMANDA L WILLIS/Primary Examiner, Art Unit 2156