DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Arguments/Amendments
Applicant's amendments filed June 23, 2026 have been fully considered.
Claim 36 is cancelled.
Claim 47 is new.
Objection to claims 43, and 45-46 is withdrawn in response to applicant’s amendments.
Applicant’s arguments with respect to claim(s) 27, and 38 have been considered but are moot because the new ground of rejection does not rely on any reference applied in the prior rejection of record for any teaching or matter specifically challenged in the argument.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 27-29, 37-38, 42, and 44, 47 are rejected under 35 U.S.C. 103 as being unpatentable over Luo et al (US 20210203627 A1, hereinafter “Luo”) in view of Gorgenyi et al. (US 20190289070 A1, hereinafter “Gorgenyi”).
Regarding claim 27,
Luo teaches:
A method (Luo: ¶209, “. . . a method 2600 to generate a message based on at least one post-processing operation on image data. . .”), comprising:
displaying, by a first electronic device, a first interface, wherein a first picture is displayed on the first interface (Luo: ¶130, “. . . An image 812, including a representation of the face of the user 802, is displayed on a display screen 852 of the client device 102. . .”; NOTE: Also see Fig. 11 and Fig. 8B. The display screen is the first interface.);
detecting, by the first electronic device, a first user operation (Luo: ¶140, “. . . when a user performs a particular touch input gesture (e.g., swipe, or drag gesture) within a user interface, a particular media overlay is identified and presented to the user. . .”);
displaying, by the first electronic device, a second interface in response to detecting the first user operation (Luo: ¶140, “. . . when a user performs a particular touch input gesture (e.g., swipe, or drag gesture) within a user interface, a particular media overlay is identified and presented to the user. . .”; NOTE: The second interface is the interface which the display screen is displaying the particular media overlay in response to the user touch input.);
wherein the second interface comprises the first picture and a first control, the first control comprises first content, and the first content is associated with a first object in the first picture (Luo: ¶133-134, “. . . FIG. 9A, an image 812 including the representation of the face of the user 802 is shown in the display screen 852 of the client device 102. A first media overlay 910 is applied to the image 812. . . . FIG. 9B, . . . graphical elements that are displayed in conjunction with the image to provide a composite image with several applied media overlays”; ¶143, “As illustrated in user interface 1100, an image 1105 corresponding to a representation of a scene with a portrait of a person, including a first media overlay 1110 (e.g., graphical text indicating a day of the week), is displayed in the display screen 852 of the client device 102”; ¶52, “. . . Examples of audio and visual content include pictures, texts, logos, animations, and sound effects. An example of a visual effect includes color overlaying. The audio and visual content or the visual effects can be applied to a media content item (e.g., a photo) at the client device 102. For example, the media overlay may include text that can be overlaid on top of a photograph taken by the client device 102. In another example, the media overlay includes an identification of a location overlay (e.g., Venice beach), a name of a live event, or a name of a merchant overlay (e.g., Beach Coffee House). In another example, the annotation system 206 uses the geolocation of the client device 102 to identify a media overlay that includes the name of a merchant at the geolocation of the client device 102. . .; NOTE: As illustrated in Fig 9A, 9B, and 11, the second interface is the interface which is the result of displaying the overlay on the first picture in response to user touch input. The first control is the overlay which comprise a first content such as text or merchant information, location, or graphical elements. Referencing to fig 9B, The first content (overlay 910) is associated with a first object in the picture which is the face of a person in the picture because the overlay is not randomly overlaid on the screen, but specifically overlaid on the eye region of the face.);
detecting, by the first electronic device, a second user operation (Luo: ¶41, “The messaging server system 108 supports various services and operations that are provided to the messaging client application 104. Such operations include transmitting data to, receiving data from, and processing data generated by the messaging client application 104. This data may include, message content, client device information, geolocation information, media annotation and overlays, message content persistence conditions, social network information, and live event information, as examples. Data exchanges within the messaging system 100 are invoked and controlled through functions available via user interfaces (UIs) of the messaging client application 104”; NOTE: The second operation is when the user sends or transmits the message which is detected by the system when invoked through functions available in the UI); and
sending, by the first electronic device, a first message to a second electronic device in response to detecting the second user operation (Luo: ¶100, “. . . the post-processing engine 620, in cooperation with the sharing engine 640, generates a message. The message includes the output frame 608, the media overlay metadata 609 and/or other metadata (e.g., as described herein), and in some instances, the original input frame 602. The sharing engine 640 transmits the generated message from the client device 102 to one or more other client devices. . .” NOTE: The first electronic device is the sender device, the second electronic device is the recipient of the transmitted message),
wherein the first message comprises the first picture and the first content (NOTE: As discussed above, the generated message comprise the first picture (output/input frame) and the first content (the media overlay metadata),
the second electronic device to display the first picture and a second control based on the first message, and the second control comprises the first content (Luo: ¶58, “. . . overlays that are displayed as overlaid on an image or video during presentation to a recipient user. . .”; ¶71, “asset identifiers 756: a set of identifiers for assets in the message with post-processing. For example, respective asset identifiers can be included for assets that are determined by the particular media overlay. In an embodiment, such assets are created by the media overlay on the sender side client device, uploaded to the messaging server application 114, and utilized on the receiver side client device in order to recreate the message”; NOTE: The recreated message in the receiver side is based on the first message because it is recreated using assets created in the sender side. Since the generated image comprise assets which include overlay metadata and images from the sender device, then the recreated message in the receiver side comprise the first content.).
However. Luo fails to teach: after sending the first message, detecting, by the first electronic device, a seventh user operation performed on the first control, wherein the seventh user operation comprises an operation of modifying the first content modifying, by the first electronic device, the first content comprised in the first control in response to detecting the seventh user operation; and sending, by the first electronic device, a request to modify the first content, wherein the request comprises an identifier of the first content, a type of the first content, and one or more synchronization parameters indicating the modified first content, the request triggers the second electronic device to synchronize the second control based on the one or more synchronization parameters.
The analogous art Gorgenyi teaches:
after sending the first message (Gorgenyi: ¶30, “. . .a user may be provided a reminder 510 in a browser window 510 that annotations are being shared. . .” ; NOTE: The user that provided the reminder message is the device sending the first message),
detecting, by the first electronic device, a seventh user operation performed on the first control (Gorgenyi: ¶32, “. . . an annotation . . . is received, e.g., at a host computer, as illustrated in FIG. 5. This annotation may comprise a highlight annotation 530, which may include highlighting some or all of the text indicated . . . permits a user to not only enter electronic ink annotations, but also to highlight text, and to enter text annotations, as well, in addition to some additional options for further annotations, such as inserting images or other graphics. . .”; ¶40, “If a text annotation is being made—such as by using a “note taking” function, e.g., as specified within the PDF specification—the coordinates where the note-taking action was invoked is received. And, subsequently, as notes are taken, e.g., in a textbox . . .”; ¶85, “. . . The input device(s) 950 may be a touch input device such as a keyboard, mouse, pen, or trackball, a voice input device, a scanning device, or another device that provides input to the computing system . .”; NOTE: The user operation that is detected by the first electronic device (host device) is the user typing on a keyboard in textbox, or highlighting texts using an input device such as a mouse. The first control is the text box where the user operates the seventh user operation, which is typing in the textbox),
wherein the seventh user operation comprises an operation of modifying the first content modifying, by the first electronic device, the first content comprised in the first control in response to detecting the seventh user operation (NOTE: In paragraph 40 as cited above, the user of the first electronic device (host device) subsequently taking notes, (the notes is the first content), and modifying it by typing to add notes in the first control, which is the textbox.);
and sending, by the first electronic device, a request to modify the first content (Gorgenyi: ¶50-51, “. . . the browser may be directed to listen to requests from a second location, such as a second IP address from which annotation synchronization requests may be generated. . . the annotations are first serialized into JSON data and, when content is being shared as described above, the same serialized data is streamed to the target computers, e.g., via UDP, so that the annotations can be synchronized in fixed digital documents situated on those devices. . .”; ¶55, “Once a synchronization request is sent out, it is received by the target computers which have been configured to receive such requests. . . the users of these devices can see the same annotations on their devices, without having to manually enter them themselves”; NOTE: A synch request is sent out by the first electronic device (the device sending the synch request) to modify the first content (annotations and modifications to the message). The synch request is the request to modify the first content so it will synch with other people),
wherein the request comprises an identifier of the first content, a type of the first content, and one or more synchronization parameters indicating the modified first content (Gorgenyi: ¶53, “A sample JSON payload for a given annotation may resemble the following:
TABLE-US-00003 { “id”: “A564gd&@”, “page”: 45, “type”: “highlight”, “topX”: 45, “topY”: 12, “width”: 123, “height”: 34 }”),
the request triggers the second electronic device to synchronize the second control based on the one or more synchronization parameters (Gorgenyi: ¶55, “Once a synchronization request is sent out, it is received by the target computers which have been configured to receive such requests. Once the message is received, it is deserialized and (if provided) its “id” attribute value is obtained. This “id” attribute may be used by the browser to determine which of the fixed digital documents that are open or may be opened using that browser on that target computer match that id.”; NOTE: The target computers constitute the second electronic device that receives the synchronization request, it deserializes the message taking into account parameters such as the id value. The second control is the textbox in the receiver device, or the target computers.).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Luo, and Gorgenyi, and include: after sending the first message, detecting, by the first electronic device, a seventh user operation performed on the first control, wherein the seventh user operation comprises an operation of modifying the first content modifying, by the first electronic device, the first content comprised in the first control in response to detecting the seventh user operation; and sending, by the first electronic device, a request to modify the first content, wherein the request comprises an identifier of the first content, a type of the first content, and one or more synchronization parameters indicating the modified first content, the request triggers the second electronic device to synchronize the second control based on the one or more synchronization parameters.
The reason for doing so is so that “the annotation received at the host computer in the fixed digital document PDF book may be applied directly and automatically to each of the target computers, and the users of these devices can see the same annotations on their devices, without having to manually enter them themselves” (Gorgenyi: ¶55).
Regarding claim 28, depending on 27,
The combination of Luo, and Gorgenyi teaches:
The method according to claim 27,
Luo further teaches:
wherein before displaying, by the first electronic device, the second interface, the method further comprises displaying one or more options (Luo: ¶188-190, “. . . selectable graphical items 1650 may be presented in a carousel arrangement in which a portion or subset of the selectable graphical items 1650 are visible on a display screen of a given computing device (e.g., the client device 102). . . the selectable graphical items correspond to respective media overlays that are applied to media content as part of post-processing. . .”; NOTE: Options are displayed to the user in the form of a carousel as illustrated in Fig. 16 and 17. These options are for which media overlays are applied to the media. The user selects from one of the options, before it is overlaid to the media content for displaying.),
and wherein displaying, by the first electronic device, the second interface comprises:
detecting a fourth user operation performed on a first option of the one or more options (Luo: ¶190, “. . . The selectable graphical item 1650 is selected via a touch input by the user”; NOTE: The first option is the object selected by the user from the selectable graphical items.);
and displaying, by the first electronic device, the first control in the first picture in response to detecting the fourth user operation, wherein a type of the first control is determined based on the first option (Luo: ¶: 185, “. . . each media overlay corresponding to a media overlay is associated with at least one classification (e.g., a category associated with a given media overlay). Classifications may correspond various categories (e.g., a classification label or category indicator for the media overlay), which are then utilized for organizing (e.g., grouping) respective media overlays into various sets (e.g., groups) for including with a user interface for presenting (and interaction) with a given user of the electronic device. Based at least in part on such groupings, the user can select multiple media overlays, via the user interface, for applying to media content during a post-processing stage as discussed with respect to FIG. 16 and FIG. 17. . .”; NOTE: A user is presented with a carousel which comprise selectable graphical items which correspond to respective media overlays. The first control that is displayed is the one that is selected by the user in the carousel in response to detecting a touch operation on the first option. Luo teaches that each of the media overlay has types (classification or category). Therefore, a type of media overlay is then determined based on the first option. ).
Regarding claim 29, depending on 28,
The combination of Luo, and Gorgenyi teaches:
The method according to claim 28,
Luo further teaches:
wherein the fourth user operation comprises an operation of selecting the first option and a second option (Luo: ¶185, “. . . the user can select multiple media overlays. . .”; ¶198-199, “. . . FIG. 17 . . . in order to apply the media overlay corresponding to the selectable graphical item 1752. To provide layering (e.g., stacking) of different media overlays, a media overlay 1758 is applied in response to the subsequent input. . . ”; NOTE: The subsequent tap to select an additional overlay is the operation selecting the second option. The first tap selects the first option, the subsequent tap to select the second option, providing a layered or stacked overlays.),
the second option is one of the one or more options (NOTE: The second option is one of the one or more options because they are within the same carousel provided to the user as illustrated in Fig. 17. ),
and wherein the second interface further comprises a third control, the third control comprises third content, the third content is associated with the first object in the first picture, and the third content is different from the first content (Luo: ¶134, “As illustrated in FIG. 9B, a second media overlay 912 is applied to the image comprising the first media overlay. The second media overlay 912 is displayed in conjunction with the first media overlay 910 in the display screen 852. In this manner, multiple media overlays can be applied to the image resulting in effects and graphical elements that are displayed in conjunction with the image to provide a composite image with several applied media overlays”; NOTE: The third control is the additional media overlay displayed to the user together with the first overlay and the face which altogether constitutes the second interface. In reference to Fig. 9B, two different media overlays are displayed (910 and 912). They are associated with the first object in the picture (face) as shown in fig. 9B).
Regarding claim 37, depending on 27,
The combination of Luo, and Gorgenyi teaches:
The method according to claim 27, wherein:
the first user operation is an operation of touching and holding the first picture; or the first user operation is an operation of tapping a fourth control on the first interface (Luo: ¶190-192, “. . . The selectable graphical item 1650 is selected via a touch input by the user. . . To apply the media overlay corresponding to the third selectable graphical item 1654, a further input may be provided by the user such as another touch input (e.g., tap, or press). . .”; NOTE: The fourth control is the graphical item selected by the user on the device interface via a tap input.)
Regarding claim 47, depending on 27,
The combination of Luo, and Gorgenyi teaches:
The method according to claim 27,
Gorgenyi further teaches:
wherein the one or more synchronization parameters comprise at least one of a replaced content of the modified first content, a size of the modified first content, a location of the modified first content, or a shape of the modified first content (Gorgenyi: ¶53, “{ “id”: “A564gd&@”, “page”: 45, “type”: “highlight”, “topX”: 45, “topY”: 12, “width”: 123, “height”: 34 }” NOTE: In reference to Table 3, paragraph 53, the payload included in the synchronization request comprise a location of the highlight including the page and page coordinates where the highlight was made.).
Regarding claim 38,
Luo further teaches:
A method, applied to a second electronic device (Luo: ¶107, “. . . the receiver side client device in order to recreate the message. . .”),
wherein the method comprises:
receiving a first message sent by a first electronic device, wherein the first message comprises a first picture and first content, the first content is associated with a first object in the first picture, and the first content is separate from the first picture (Luo: ¶107-110, “. . . For example, respective asset identifiers can be included for assets that are determined by the particular media overlay. In an embodiment, such assets are created by the media overlay on the sender side client device, uploaded to the messaging server application 114, and utilized on the receiver side client device in order to recreate the message. . .”; NOTE: As discussed in the rejection of claim 27, the first electronic device, which is the sender’s device creates a message comprising a first picture and first content associated with a first object in the first picture, and the first content is separate from the first picture, (image of an object/face with media overlays) >> the message is then recreated in the second electronic device, which is the receiver. Recreating a message will result in the recreation of the first picture and the first content that was sent by the sender);
displaying, by the second electronic device, a third interface based on the first message, wherein the first picture and a second control are displayed on the third interface, and the second control comprises the first content (NOTE: The third interface is the interface of the receiver’s device. Because the message is recreated, the first control comprising the first content is recreated on the receiver device as the second control comprising the first content. Because the message included the first picture, once it is recreated in the receiver’s client device, it will also be displayed.).
However Luo fails to teach: after receiving the first message, receiving, by the second electronic device, a second message indicating that the first content has been modified, wherein the second message comprises an identifier of the first content, a type of the first content, and one or more synchronization parameters indicating the modified first content; and synchronizing, by the second electronic device, the second control based on the one or more synchronization parameters.
The analogous art Gorgenyi teaches:
after receiving the first message, receiving, by the second electronic device, a second message indicating that the first content has been modified, wherein the second message comprises an identifier of the first content, a type of the first content, and one or more synchronization parameters indicating the modified first content; and synchronizing, by the second electronic device, the second control based on the one or more synchronization parameters. (Gorgenyi: ¶53, “A sample JSON payload for a given annotation may resemble the following: TABLE-US-00003 { “id”: “A564gd&@”, “page”: 45, “type”: “highlight”, “topX”: 45, “topY”: 12, “width”: 123, “height”: 34 }”),
Gorgenyi: ¶55, “Once a synchronization request is sent out, it is received by the target computers which have been configured to receive such requests. Once the message is received, it is deserialized and (if provided) its “id” attribute value is obtained. This “id” attribute may be used by the browser to determine which of the fixed digital documents that are open or may be opened using that browser on that target computer match that id.”; NOTE: The target computers constitute the second electronic device that receives the synchronization request, it deserializes the message taking into account parameters such as the id value. The second control is the textbox in the receiver device, or the target computers. After receiving the first message corresponds to the unmodified first content that was previously shared. As discussed in the rejection of claim 27 in regard to the host device, the host device modifies the first content, for example, adding annotations, this creates the second message (synchronization request as discussed in the rejection of claim 27) and is serialized with the id,type, and other parameters in the JSON shown in Table 3, paragraph 53. The synchronization request that is received by the target computer, which is the second electronic device deserializes it. In the example in Table 3, the highlight with its coordinates is the modification done to the first content).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Luo, and Gorgenyi, and include: after receiving the first message, receiving, by the second electronic device, a second message indicating that the first content has been modified, wherein the second message comprises an identifier of the first content, a type of the first content, and one or more synchronization parameters indicating the modified first content; and synchronizing, by the second electronic device, the second control based on the one or more synchronization parameters.
The reason for doing so is so that “the annotation received at the host computer in the fixed digital document PDF book may be applied directly and automatically to each of the target computers, and the users of these devices can see the same annotations on their devices, without having to manually enter them themselves” (Gorgenyi: ¶55).
Regarding claim 42, depending on 38,
Luo further teaches that the second electronic device, which is the receiver has the same functionality and the same instance of the client device used by the sender as disclosed in paragraph 38. Therefore, the second electronic device has the same user to UI functionalities, capabilities, and logic.
Method of claim 42 is drawn to the method applied in a second electronic having the same control and content limitations as used in claim 29 and is rejected for the same reasons of anticipation as used above.
Regarding claim 44, depending on 38,
Method of claim 44 is drawn to the method applied in a second electronic corresponding to the same remote modification limitations as used in claim 36 and is rejected for the same reasons of anticipation as used above.
Claims 30-35, 39-41, 43, and 45-46 are rejected under 35 U.S.C. 103 as being unpatentable over Luo in view Gorgenyi further in view of Rathod (US 20220179665 A1, hereinafter “Rathod”).
Regarding claim 30, depending on 29,
The combination of Luo, and Gorgenyi teaches:
The method according to claim 29,
Although Luo teaches displaying overlays with contextual information and activating media overlays for specific purposes as disclosed in paragraph 204, Luo does not teach a user operation performed on the media overlay to display further content associated with the content for that overlay object. Therefore, Luo fails to teach: wherein before detecting, by the first electronic device, the second user operation, the method further comprises: detecting, by the first electronic device, a fifth user operation performed on the first control; and displaying, by the first electronic device, fourth content in response to the fifth user operation, wherein the fourth content is associated with the first content.
The analogous art Rathod teaches:
A method for displaying contextual keywords with corresponding associated or related or contextual one or more types of user actions, reactions, call-to-actions, relations, activities and interactions for user selection, wherein displaying contextual keywords based on plurality of factors and in the event of selection of keywords store and associate keywords with unique identity of user and in the event of selection of keyword associated action, associate keywords with unique identity of user and store data related to selected type of user actions, reactions, call-to-actions, relations, activities and interactions (Rathod Abstract.)
Rathod further teaches tapping on a first control to view further content (Rathod Fig. 66A, ¶759, “. . . interfaces related to one or more advertisements e.g. 6605 (6607, 6609, 6611, 6613, 6615, 6617, 6620, 6622 & 6624) at user interface 6605 on user display 210 on user device 200 and enabling user to access said one or more presented contextual augmented reality applications, functions, controls (e.g. button), web services, objects, interfaces. . .”; ¶764, “FIG. 66 . . . user tap on button . . . view catalogues, offers, sale, and discounts 6613. . . ” NOTE: tapping button 6613 let’s the user view further content catalogues, offers, sale, and discounts associated. The other buttons also lets the user access different contextual information related to the image).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Luo, Gorgenyi and Rathod and include: wherein before detecting, by the first electronic device, the second user operation, the method further comprises: detecting, by the first electronic device, a fifth user operation performed on the first control (NOTE: Rathod user tapping on the first control button 6613); and displaying, by the first electronic device, fourth content in response to the fifth user operation (NOTE: The fourth content are the catalogues, offers, sale, and discounts for viewing), wherein the fourth content is associated with the first content (NOTE: The first content is the text information of the button, which is directly associated to the content for viewing when the user tap on the button.)
The reason for doing so is to provide and for “displaying contextual keywords with corresponding associated or related or contextual one or more types of user actions, reactions, call-to-actions, relations, activities and interactions for user selection” (Rathod: ¶2).
Regarding claim 31, depending on 30,
The combination of Luo, Gorgenyi, and Rathod teaches:
The method according to claim 30,
Although Luo teaches generating the first message including overlay metadata before transmission to a receiver client, Luo fails to teach wherein the first message comprises a first web address, and the fourth content is content comprised in a page corresponding to the first web address.
Rathod further teaches a web address that can be accessed by the user and accessing websites associated with the URL to view advertisements (Rathod: ¶341, “FIG. 16-17 . . . create one or more advertisement campaigns . . . links. . . where advertisements present to users or viewers . . . provide advertisement group name, . . . and links or Uniform Resource Locator (URL) . . . which will show to target criteria specific viewing users . . .; ¶876, “. . . advertises can create session and during session advertise one or more applications (enable viewers during session to view details, download, try, install, register, refer, purchase, purchase in app features or enhanced features), web sites (enable advertiser to building brand, sell or provide information about availability of curate or quality or new types of products and services to viewers during session. . .”).
It would have been obvious to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Luo, Gorgenyi, and Rathod and include: wherein the first message comprises a first web address, and the fourth content is content comprised in a page corresponding to the first web address.
The reason for doing so is to “enable viewers during session to view details, download, try, install, register, refer, purchase, purchase in app features or enhanced features” (Rathod: ¶341)
Regarding claim 32, depending on 30,
The combination of Luo, Gorgenyi, and Rathod teaches:
The method according to claim 30,
Rathod further teaches:
viewing encyclopedia content information of users’ surrounding in a mobile camera view (Rathod: ¶144, “. . . displays information about users' surroundings in a mobile camera view. . . . accesses the Wikipedia data set to provide geographic information (e.g. longitude, latitude, distance), history, and contact details of points of interest. . . overlays . . . its information. . .), additionally, Rathod teaches URL or web address for shopping as described in paragraph 341. Rathod further teaches providing and displaying to a user, contextual information of a particular object from captured images or videos of a scene when an option is selected (¶766, “Like photo icon 6627 for taking photo, video icon 6628 for recording video, now user is able to capture or scan view or scene or particular object . . . retrieves . . . contextual contents or information (web links, . . . news, article . . . including photo(s), video(s)).
It would have been an obvious design choice to display specific contextual information when a user selects an option among Rathod’s list of contextual information to a person having ordinary skill in the art (PHOSITA) before the effective filing date of the claimed invention to combine Luo, and Rathod to include include: wherein when the first option is an encyclopedia, the fourth content comprises an encyclopedia introduction of the first object; when the first option is a picture, the fourth content comprises a picture whose image content is the first object; when the first option is a video, the fourth content comprises a video whose video content is the first object; when the first option is information, the fourth content comprises news information that introduces the first object; or when the first option is a source, the first content comprises a second web address, the first electronic device obtains the first picture based from the second web address, and the fourth content is content comprised in a web page indicated by the second web address
The reason for doing so is to provide and for “displaying contextual keywords with corresponding associated or related or contextual one or more types of user actions, reactions, call-to-actions, relations, activities and interactions for user selection” (Rathod: ¶2) and to provide the users contextual information for their output of choice.
Regarding claim 33, depending on 32,
The combination of Luo, Gorgenyi, and Rathod teaches:
The method according to claim 32,
Although Luo teaches media overlay options (FIG. 16 1650), and media overlays comprising a deep link (Luo ¶204), Luo fails to teach: wherein if the first option is a source, the first content comprises a second web address, the second web address is a web address provided by shopping software, and the first picture is an image comprised in a page indicated by the second web address.
Rathod further teaches:
wherein if the first option is a source, the first content comprises a second web address, the second web address is a web address provided by shopping software, (Rathod: ¶341, “create one or more advertisement campaigns including . . . and links or Uniform Resource Locator (URL) 1630, . . .”); ¶756 “. . identifies, matches, recognizes and detects said scanned object or object inside captured photo or video . . . when user tap on “Stories” button 6607 then system searches, matches, identifies said advertised shop or brand or product 6501. . . and presents said matched visual media items e.g. 6688 at use interface or display 210 or user device 200. . . ”; ¶764 - 765, “. . . In another example user can scan object or product and search & view information about it 6611. . . view catalogues, offers, sale, and discounts . . . can download & install associated application 6620 . . can make order, make payment and buy and add to cart product . . . by tapping on augmented sharing button (e.g. 6629 or 6625) to enabling user to capture visual media and/or retrieve . . . scanned object(s) . . . based on object recognition technologies (which identifies object(s) related keywords via server module 180 and server module 180 searches, matches said identified keywords specific information from one or more sources including one or more web sites (search engines, social networks), applications. . .; ¶757, “In one embodiment advertiser . . . provide . . . control (e.g. button), web service, object, interface. . .”; NOTE: The first option is the button 6624 as shown in Fig. 66. This option is a source because it leads the user to the website displaying items recognized in the first picture and then displayed in another user interface of a shop’s website, the source. Rathod further teaches that advertisers provide web services with associated URL or web address and associated shopping software or application for download. ).
and the first picture is an image comprised in a page indicated by the second web address (NOTE: Referencing to fig. 66, 66A-66B, the first picture is the picture with the dress in fig. 66A, user taps button 6607 and shown a user interface of a carousel of the dresses for shop products matching the first picture.)
Regarding claim 34, depending on 30,
The combination of Luo, Gorgenyi, and Rathod teaches:
The method according to claim 30,
Although Luo teaches media overlays comprising contextual information, such as travel information corresponding to a location based on the geolocation of the client device and other indicia associated with a merchant as described in paragraph 52, Luo fails to teach: wherein: the first control comprises travel information corresponding to a first location and a second location, wherein the first location is related to the first object, the second location is related to the first electronic device, and the first content comprises the first location; and the second control comprises travel information corresponding to a third location and the first location, and the third location is related to the second electronic device.
Rathod further teaches:
wherein:
the first control comprises travel information corresponding to a first location and a second location, wherein the first location is related to the first object, the second location is related to the first electronic device, and the first content comprises the first location; (Rathod: ¶815, “The application manager 115 can receive . . . location information 147, and other information (such as user information 151) to configure content that is to be provided by the UI component 120. . . output to a display . . . features 121 can be area-specific (e.g., based on the current location of the computing device) to display information that is particular to the area. The user interface features 121 can also provide dynamically updated content based on user selections provided via the user input 111”; ¶835, “. . . for an on-demand transport service, the summary panel can provide area-specific information, such as the estimated time of arrival for shooting location or point of interest location or pickup”; NOTE: The first control is the option that the user selects to display the summary panel providing content for display to the user. The summary panel displays a first location related to the first object which is the point of interest location or pickup. The second location corresponding to the first electronic device is the location information also displayed based on the current location of the user’s electronic device. First location = location of interest, second location = location information of the user’s device based on its current location; user taps on first control >> display summary panel with ETA information based on the first and second location. Therefore, the first control comprises travel information such as ETA corresponding to the first location (point of interest location) and a second location (user’s device current location))
and the second control comprises travel information corresponding to a third location and the first location, and the third location is related to the second electronic device (Rathod: ¶244, “Gateway 120 may be configured to send and receive user contents or posts or data to targeted or prospective, matched & contextual viewers based on preferences, wherein user data comprises . . . status, updates, presence information, locations, . . .”; NOTE: The second control is the media overlay/button on the receiver’s device. The third location is the location of the receiver/viewer device that can view content that is matched and contextual based on the viewer’s current location. Sender sends message that comprises the first location (location of interest) to a receiver that views the message >> the viewer’s device can view data corresponding to the viewer’s device location since it is matched and contextual based on preferences including location information of the viewer’s device.).
Regarding claim 35, depending on 34,
The combination of Luo and Rathod teaches:
The method according to claim 34,
wherein: the travel information comprises at least one of the following travel options: a train, a flight, or a bus (Rathod: ¶876, “. . . enable viewers during session to book tickets of . . . tours & travels (packages, flights, cruises). . .; user selected or searched or select from categories . . . and auto present to user day to day activities . . . travel service (cab, bus etc.; NOTE: also see Fig. 98),
the fifth user operation is an operation performed on a third option in the at least one travel option (Rathod: Fig. 98, ¶990, “FIG. 98. . . graphical user interface enabling user to browse 9805, search 9803 and select category (e.g. 9820) and sub-category . . . and share with one or more contacts and/or destinations. Directory comprises type or categories”; NOTE: In reference to Fig. 98, A GUI with options is displayed and enabling the user to perform selections by tapping as the fifth user operation on a third option, which is the UI 9850. First option is the “travel” option >> second option is the “tourist places” option >> and third option is “USA” option, which results in a fourth content 9870 which is a table of places.);
and the fourth content comprises a timetable and a ticket price that are of the second option and that are between the third location and the first location (Rathod: ¶768, “. . . showing of one or more types of retrieved content . . . map (show place point, route, estimated time to reach etc.), one or more types of statistics & analytics, attributes or features or characteristics related information, price or fees or payment information. . .” ¶820, “. . . In many instances, a geographic area that is specific to the user can be based on the user's current location. . .”; NOTE: Also see Fig. 78C showing a timetable for the London Tour package with dates and a price “72,990”. Additionally, fig. 79 also illustrates a timetable. Referencing to Fig 98, it is shown that there is a first option travel >> second option packages >> third option would show content regarding price or payment information for selection >> displays package detail such as shown in fig 78C. Since the data is matched and contextual based on who is viewing the message, in this case the receiver in the third location, the data that will be displayed will be the timetable and a ticket price between the third location, which is the receiver’s current location, and the location of interest, which is London.)),
and the timetable comprises a shift number, a departure location, a destination, a departure time, or an arrival time (Rathod: Fig. 78C; ¶835, “. . . the summary panel can provide area-specific information, such as the estimated time of arrival. . .”; NOTE: Fig. 78C shows a timetable with dates and destination.).
Regarding claim 39, depending on 38,
The combination of Luo, and Gorgenyi teaches:
The method according to claim 38,
Luo further teaches that the second electronic device, which is the receiver has the same functionality and the same instance of the client device used by the sender as disclosed in paragraph 38. Therefore, the second electronic device has the same user to UI functionalities, capabilities, and logic.
The limitations of claim 39 is drawn to the method applied in a second electronic device having the same user interaction as the “fifth user operation” in claim 30 and is rejected for the same reasons of obviousness as used above.
Regarding claim 40, depending on 39,
Method of claim 40 is drawn to the method applied in a second electronic corresponding to the second electronic device as used in claim 34 and is rejected for the same reasons of obviousness as used above.
Regarding claim 41, depending on 40,
Method of claim 41 is drawn to the method applied in a second electronic corresponding to the same limitations as used in claim 35 and is rejected for the same reasons of obviousness as used above.
Regarding claim 43, depending on 38,
Method of claim 41 is drawn to the method applied in a second electronic having the same web address and page content limitations as in claim 31 and is rejected for the same reasons of obviousness as used above.
Regarding claim 45, depending on 38,
Method of claim 45 is drawn to the method applied in a second electronic having the same type-based content limitations as in claim 32 and is rejected for the same reasons of obviousness as used above.
Regarding claim 46, depending on 38,
Method of claim 46 is drawn to the method applied in a second electronic having the same shopping source and software limitations as in claim 33 and is rejected for the same reasons of obviousness as used above.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PATRICK GALERA whose telephone number is (571)272-5070. The examiner can normally be reached Mon-Fri 0800-1700 ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, King Poon can be reached at 571-270-0728. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PATRICK P GALERA/Examiner, Art Unit 2617 /KING Y POON/Supervisory Patent Examiner, Art Unit 2617
Prosecution Insights