Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
DETAILED ACTION
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), that are applied for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-3, 7, 10, 13, 19-22, 26-28, 32, 35, 38, 44-47 are rejected under 35 U.S.C. 103 as being unpatentable over Ganong et al. (US Pub 2018/0046855 A1) in view of Greenberger et al. (US Pub 2018/0101973 A1).
As to claim 1, Ganong discloses a method, comprising:
analyzing a subject image to detect a first person and a second person depicted in the subject image (¶0018, “enabling at least one of the computer terminals to initiate a face recognition routine on the digital image, the face recognition routine producing a list of one or more persons whose faces are depicted in the digital image, at least one of the persons being one of the individuals” ¶0138, “generating face signatures based on faces depicted in images.”);
identifying at least one of the first person or the second person in at least one stored image of a plurality of stored images, wherein the plurality of stored images is associated with a plurality of persons (¶0042, “determining a count of a total number of digital images where the identified person appears; for each identified person shown in at least one of the digital images with the selected identified person, determining a count of a total number of digital images where the respective identified person appears with the selected identified person” ¶0146, “Every previously identified face for every known person may be compared with each new face processed by the system. When viewing the faces related to a specific known person, any suspected matches generated by the invention may be displayed and the user may be asked to confirm that the matches are correct.” ¶0225, “Each portrait image may be associated with an identified person shown in the respective portrait. The identification of the person may be stored in the database, another database, or in metadata associated with the respective portrait image. The at least one computer may display the respective portrait of at least one identified person associated with a user.”);
determining a relationship between the first person and the second person in the subject image based at least in part on information corresponding to the at least one stored image (¶0110, “The database may also contain metadata for photos and people as well as relationships between known persons and the associated face images.” ¶0210, “Where a digital image shows more than one person, the one or more computers may associate relationships between identified persons based at least partly on respective identified persons being included in the digital image.”);
associating an indication of the determined relationship with metadata of the subject image (¶0039, “the database comprising a plurality of portraits, each portrait associated with an identified person shown in the respective portrait, the method comprising: displaying the respective portrait of at least one identified person associated with a user; displaying a visual representation of at least one personal relationship to the user; assigning at least one of the displayed portraits to at least one of the displayed personal relationships, in accordance with a received user input; and storing the personal relationship assignments in the database.” ¶0042, “a database of digital images and respective metadata identifying a name of at least one identified person shown in the respective digital image” ¶0163, “FIG. 9 shows face images for known persons plus the checkboxes for applying Boolean searching such as AND, OR, and NOT selections associated with names of known persons or metadata related to images. A novel feature of the invention is the ability to select photos in a visual manner by allowing the user to click on a thumbnail view of the faces of known persons (59), and applying Boolean operations (61) for each face enabled by checkboxes. This aspect of the GUI enables the creation of an album by combining various search criteria and filters that are applied against the total photo and face database.” Fig. 28, ¶0222, “each user of the system of the present invention may have six basic relationships of mother, father, sibling, spouse, daughter and son, as shown in template 28b. As the user drags portraits on to the family tree the family tree grows to show the new nodes. Optionally new blank nodes are added for the common relationships to the selected node. As more persons are added the family tree expands as users are added and relationships are defined. When the user has dragged all portraits the family tree is completed by eliminating extraneous relationship or persons such as friends and colleagues. Users will likely have friend and colleague relationships that are pertinent to their personal social map but are identified in a modified network map using a similar drag and drop method.” ¶0224, “other types of organizational structures representing relationships between persons may be presented in a chart format for populating by dragging portraits thereto” ¶0232, “When a set of digital photos contains name tags that identify the people represented in those photos then there is potential to “mine” that information and generate potentially interesting, entertaining, and useful techniques for displaying relationships between people that have been tagged in those photos.”); and
generating, (¶0225, “The at least one computer may display a visual representation of at least one personal relationship to the user.” “The at least one computer may assign at least one of the displayed portraits to at least one of the displayed personal relationships, in accordance with a received user input. The at least one computer may store the personal relationship assignments in the database. The visual representation may include a representation of a tree organizational structure with a plurality of tree nodes, such as for a family tree, where each tree node corresponds to one of the at least one personal relationships.” ¶0232, “generate potentially interesting, entertaining, and useful techniques for displaying relationships between people that have been tagged in those photos.” ¶0237, “show the relationships purely in list format based on the tiers away from the central person.” ¶0258, “determine at least one first identified person shown in at least one of the digital images together with the selected identified person, and display a visual representation. The visual representation may include, for each first identified person, a first tier node representing the selected identified person and the respective first identified person being shown in at least one of the digital images together.”).
Ganong does not explicitly discloses generating, for simultaneous display with the subject image, a visual identifier.
Greenberger teaches generating, for simultaneous display with the subject image, a visual identifier that indicates the determined relationship between the first person and second person (Greenberger, Fig. 3-4, abstract, “A relationship type and a relationship strength between two or more people identified in a first image is determined based on profile data corresponding to each of the two or more people and a set of data elements determined to be common to the profile data corresponding to each of the two or more people. An image overlay is selected to apply to the first image based on the relationship type and the relationship strength between the two or more people and the set of data elements determined to be common to the profile data corresponding to each of the two or more people. The image overlay is applied to the first image generating a second image that includes the first image and the applied image overlay.)
Ganong and Greenberger are considered to be analogous art because all pertain to image processing. It would have been obvious before the effective filing date of the claimed invention to have modified Ganong with the features of “generating, for simultaneous display with the subject image, a visual identifier that indicates the determined relationship between the first person and second person” as taught by Greenberger. The suggestion/motivation would have been in order to applying an image overlay to an image based on a relationship between the people identified in the image (Greenberger, ¶0001).
As to claim 2, claim 1 is incorporated and the combination of Ganong and Greenberger discloses the plurality of stored images comprises a profile picture for each of a plurality of profiles of a social media platform (Greenberger, ¶0024, “clients 110, 112, and 114 may access data posted on one or more social media websites and/or data stored in user profiles to identify relationships between the people in the captured image.”);
at least one of the first person or the second person of the subject image is identified in at least one profile picture (Greenberger, ¶0034, “image overlay manager 220 utilizes profile analyzing component 232 to help identify the relationships between people 246 in image 222.”); and
the information corresponding to the at least one stored image comprises information from a profile of the plurality of profiles that corresponds to the at least one profile picture (Greenberger, ¶0034, “Profile 224 represents a collection of data corresponding to a user of data processing system 200. The data contained in profile 224 may include, for example, names of people 248 that the user knows and is associated with. Profile 224 also includes relationships 250, which represent how the user is associated with people 248.”).
As to claim 3, claim 2 is incorporated and the combination of Ganong and Greenberger discloses the profile of the plurality of profiles that corresponds to the at least one profile picture is a first profile (Greenberger, ¶0024, “clients 110, 112, and 114 may access data posted on one or more social media websites and/or data stored in user profiles to identify relationships between the people in the captured image.”);
the plurality of stored images comprises non-profile pictures for at least a portion of the plurality of profiles of the social media platform (Greenberger, ¶0035, “Social media search component 234 searches one or more social media websites to find relationship and life event data posted on the social media websites by people 246, which were identified by facial recognition component 230, in image 222.” Event data posted on social media is non-profile pictures.);
the identifying at least one of the first person or the second person comprises (i) identifying the first person in the at least one profile picture and (ii) determining that the profile pictures of the social media platform do not comprise the second person (Greenberger, Fig. 2, ¶0025, “profiles corresponding to each of the registered users” ¶0035, “Social media search component 234 searches one or more social media websites to find relationship and life event data posted on the social media websites by people 246, which were identified by facial recognition component 230, in image 222.” Life event data is non profile picture for the second person.);
the method further comprises, based at least in part on the determining that the profile pictures of the social media platform do not comprise the second person (Greenberger, ¶0032, “facial recognition component 230, profile analyzing component 232, social media search component 234, relationship analyzing component 236”),
identifying the second person in at least one stored image of the non-profile pictures (Greenberger, ¶0034-0035, “utilizes social media search component 234 to help identify the relationships between people 246 in image 222. Social media search component 234 searches one or more social media websites to find relationship and life event data posted on the social media websites by people 246, which were identified by facial recognition component 230, in image 222.”); and
wherein the determining the relationship between the first person and the second person in the subject image is further based on information from a second profile of the plurality of profiles that corresponds to the at least one stored image of the non-profile pictures (Greenberger, ¶0035, “Image overlay manager 220 utilizes relationship analyzing component 236 to analyze the relationship and life event data found by profile analyzing component 232 in profile 224 and the relationship and life event data found by social media search component 234 in the social media websites.”.
4.-6. (Canceled)
As to claim 7, claim 1 is incorporated and the combination of Ganong and Greenberger discloses
the identifying at least one of the first person or the second person comprises identifying the first person and not the second person in the at least one stored image of the plurality of stored images (Ganong, ¶0140, “FIG. 4, the known faces may be identified with check marks (41) and the unknown faces with the symbol “X” (43).” ¶0143, “facilitates an optimal training stage by ordering the unknown faces such that the user can identify groups of detected faces that are most likely associated with a single individual.”);
the information corresponding to the at least one stored image is information associated with the first person (Ganong, ¶0141, “provides a list of known persons. If the face signature (42) corresponding to a detected face is associated with a person listed in the known persons list, the GUI may indicate such an association to the user using a graphic notation on or around the image. Otherwise, the GUI may indicate that there is no such association to the user using another graphical notation on or around the image.”);
the method further comprises determining information about the second person based at least in part on at least one of (i) characteristics of the subject image or (ii) the information associated with the first person (Ganong, Fig. 5, ¶0148, “the user may continue to associate unknown faces with known persons” ¶0267, “face comparisons may be made, and then the results may be combined by arithmetic mean to form one or more aggregate results, one for each group of similar faces.”¶0281, “interface with a database of digital images and respective metadata identifying a date of the respective digital image, a plurality of the digital images showing at least one respective unidentified person (“unidentified digital images”). The at least one computer may sort the unidentified digital images by the respective date metadata, and assign a respective clustering token to each of the unidentified digital images. The assigning may include, in accordance with a determination that a subset of the unidentified digital images each show a common unidentified person, assigning a common respective clustering token to each of the unidentified digital images of the subset. The at least one computer may group the unidentified digital images by respective clustering token. At some point, the at least one computer may receive a new digital image from another computer, database, user, or from anywhere else. The new digital image and respective metadata may identify a date of the respective new digital image, and the new digital image may include a new unidentified person. The at least one computer may then attempt to identify the new unidentified person in the received image by using or leveraging any of the clustering techniques described herein.” ¶0284, “enhance the accuracy of recognition results beyond the pure mathematics of analyzing and comparing pixels contained in the image. By taking advantage of image related metadata including date taken, camera type, location coordinates, and event information it is possible to reduce false positive data generated from the face recognition algorithms.”); and
wherein the determining the relationship between the first person and the second person in the subject image is further based on the determined information about the second person (Ganong, ¶0222, “each user of the system of the present invention may have six basic relationships of mother, father, sibling, spouse, daughter and son, as shown in template 28b. As the user drags portraits on to the family tree the family tree grows to show the new nodes. Optionally new blank nodes are added for the common relationships to the selected node. As more persons are added the family tree expands as users are added and relationships are defined. When the user has dragged all portraits the family tree is completed by eliminating extraneous relationship or persons such as friends and colleagues. Users will likely have friend and colleague relationships that are pertinent to their personal social map but are identified in a modified network map using a similar drag and drop method.” ¶0282, “In FIG. 53, when an unknown face is submitted to the face recognition method of the present invention, the recognition algorithm of the present invention may compare the unknown face with each cluster for each known person separately.”).
8.-9. (Canceled)
As to claim 10, claim 1 is incorporated and the combination of Ganong and Greenberger discloses the plurality of stored images are a first plurality of stored images; the method further comprises: determining the second person is not in the first plurality of stored images (Ganong, ¶0281, “include or interface with a database of digital images and respective metadata identifying a date of the respective digital image, a plurality of the digital images showing at least one respective unidentified person (“unidentified digital images”). The at least one computer may sort the unidentified digital images by the respective date metadata, and assign a respective clustering token to each of the unidentified digital images. The assigning may include, in accordance with a determination that a subset of the unidentified digital images each show a common unidentified person, assigning a common respective clustering token to each of the unidentified digital images of the subset. The at least one computer may group the unidentified digital images by respective clustering token. At some point, the at least one computer may receive a new digital image from another computer, database, user, or from anywhere else. The new digital image and respective metadata may identify a date of the respective new digital image, and the new digital image may include a new unidentified person. The at least one computer may then attempt to identify the new unidentified person in the received image by using or leveraging any of the clustering techniques described herein.”); and
based at least in part on determining the second person is not in the first plurality of stored images, identifying the second person in at least one stored image of a second plurality of stored images (Ganong, ¶0281, “include or interface with a database of digital images and respective metadata identifying a date of the respective digital image, a plurality of the digital images showing at least one respective unidentified person (“unidentified digital images”). The at least one computer may sort the unidentified digital images by the respective date metadata, and assign a respective clustering token to each of the unidentified digital images. The assigning may include, in accordance with a determination that a subset of the unidentified digital images each show a common unidentified person, assigning a common respective clustering token to each of the unidentified digital images of the subset. The at least one computer may group the unidentified digital images by respective clustering token. At some point, the at least one computer may receive a new digital image from another computer, database, user, or from anywhere else. The new digital image and respective metadata may identify a date of the respective new digital image, and the new digital image may include a new unidentified person. The at least one computer may then attempt to identify the new unidentified person in the received image by using or leveraging any of the clustering techniques described herein.”); and
wherein the determining the relationship between the first person and the second person is further based on information corresponding to the at least one stored image of the second plurality of stored images (Ganong, ¶0281, “At some point, the at least one computer may receive a new digital image from another computer, database, user, or from anywhere else. The new digital image and respective metadata may identify a date of the respective new digital image, and the new digital image may include a new unidentified person. The at least one computer may then attempt to identify the new unidentified person in the received image by using or leveraging any of the clustering techniques described herein. In particular, the at least one computer may perform at least one comparison of the new unidentified person to the at least one respective unidentified person of the plurality of the digital images in an order, wherein for each group of unidentified digital images, the at least one computer may perform only a single comparison of the new unidentified person to the respective common unidentified person.” ¶0282, “In FIG. 52, a recognition algorithm in accordance with the present invention may the groups of faces of a known person into one or more clusters of faces of the known person. The splitting into clusters of like faces may be based at least partly on the face signature distance between each face in the group of faces of the known person. In FIG. 53, when an unknown face is submitted to the face recognition method of the present invention, the recognition algorithm of the present invention may compare the unknown face with each cluster for each known person separately.”).
11.-12. (Canceled)
As to claim 13, claim 1 is incorporated and the combination of Ganong and Greenberger discloses
the relationship between the first person and the second person in the subject image is a first relationship determined at a first time (Ganong, ¶0033, “maintains a database of people in the photos and the relationships between people.”), the method further comprising:
determining a second relationship between the first person and the second person based at least in part on information corresponding to the at least one stored image at a second time later than the first time, wherein the information from the second time is different from the information from the first time (Ganong, ¶0034, “The computer program will dynamically spawn additional relationships as the user drags new portraits of relatives on to the family tree diagram. Once all portraits for a given family have been added to the family tree it is complete and then becomes a dynamic index for the consumer to display and locate photos. Optionally the same tree format can be used to connect friends together and show the (self-defined) relationships between different categories of friends. Example would be co-workers, high school friends, college friends, etc.” ¶0225, “The user may then specify a relationship for the newly added portrait, or a default relationship may be assigned, or the at least one computer may attempt to determine an appropriate relationship for the new relationship based at least partly on data found on a social network system to which the user is a member.”); and
updating the metadata of the subject image with an indication of the determined second relationship (Ganong, ¶0110, “The database may also contain metadata for photos and people as well as relationships between known persons and the associated face images.” ¶0235, “a database of digital images and respective metadata identifying a name of at least one identified person shown in the respective digital image.” ¶0248, “enable the visualization of data relationships extracted from photo metadata—specifically name tags and dates that are attached to digital photos.”).
14.-18. (Canceled)
As to claim 20, claim 1 is incorporated and the combination of Ganong and Greenberger discloses the generating, for simultaneous display with the subject image, the visual identifier comprises modifying, based at least in part on the determined relationship between the first person and second person, an appearance of at least one of the first or second person depicted in the subject image (Ganong, ¶0030, “The computer program may select and merge a portrait with an advertisement or a product image for display to the consumer. The computer program may be configured to adjust the size of the portrait to match the size requirements of the advertisement or product image” ¶0031, “The computer program may be configured to adjust the size of the portrait to match the size requirements of the selected face size to hide. The computer program may store the modifications in metadata or apply the changes permanently to the original photo. The edges of the overlayed (replacement) image may be adjusted to match color, intensity, brightness, texture and other characteristics to blend into the original image and be more visually appealing.” ¶0215, “When a user wants to hide negative memories one aspect of the present invention may matche faces in the face database 34a to be hidden in the photos from the photo database 34d with an image that is selected or provided by a user which is stored in the negative memory image database 34b.”).
As to claim 21, claim 1 is incorporated and the combination of Ganong and Greenberger discloses associating an indication of the determined relationship with metadata of the subject image comprises modifying the metadata to include any one or more of (i) an indication of one or more portions of the subject image that comprise at least one of the depiction of the first or second person, (ii) an identity of at least one of the first person or the second person, (iii) the information corresponding to the at least one stored image, or (iv) information about a type of the visual identifier (Ganong, ¶0031, “The computer program may be configured to adjust the size of the portrait to match the size requirements of the selected face size to hide. The computer program may store the modifications in metadata or apply the changes permanently to the original photo.” ¶0110, “A database (such as a SQL database, for example) that may be located on a user's computer or on a remote computer or cloud computer, and may contain the results of the face detection, eye detection and face recognition steps described below. The database may also contain metadata for photos and people as well as relationships between known persons and the associated face images.” ¶0142, “enable the user to enter data related to that person such as name, email address and other details, which may collectively be referred to as metadata corresponding to the person” ¶0216, “This may be accomplished by directly modifying the respective digital image(s) or updating associated metadata or other information or data to cause a display of the respective digital image(s) in a non-destructive manner such that the original source digital image is not permanently modified in the database. Accordingly, as mask may be applied to the digital image permanently, or the masking may involve modifying metadata of the digital image to cause the digital image to be masked when displayed.” ¶0225, “The identification of the person may be stored in the database, another database, or in metadata associated with the respective portrait image.”).
As to claim 22, claim 1 is incorporated and the combination of Ganong and Greenberger discloses:
receiving, from a user device, a request for relationship information between two persons depicted in a particular image; determining the particular image is the subject image, and the two persons are the first and second person; and providing, to the user device, the metadata without the subject image (Ganong, ¶0017, “(a) linking a plurality of computer terminals to a computer network and a plurality of cloud services with the digital images and metadata stored in a cloud-based data repository. (b) linking the digital image to at least one of the computer terminals; (c) enabling at least one of the computer terminals to initiate a face recognition routine on the digital image, the face recognition routine producing a list of one or more persons whose faces are depicted in the digital image, at least one of the persons being one of the individuals; and (d) enabling at least one of the computer terminals to initiate a sharing routine for disseminating the digital image to the computer terminals associated with the one or more persons.” ¶0043, “interfacing with a database of digital images and respective metadata identifying a name of at least one identified person shown in the respective digital image” ¶0045-0046. ¶0130-0133, “the computer program run by the first user (13) and the second user (17) can now exchange photos as well as metadata about those photos and about known persons” ¶0109, “transmit photos and metadata related to those photos to other users or to third-party websites the meta data can be stored in the EXIF or similar file header or be embedded inside the jpg or similar image file format in a manner similar to stenographic techniques (25) such as FLICKR™ and FACEBOOK™.”).
23.-25. (Canceled)
As to claim 26, the combination of Ganong and Greenberger discloses a system, comprising: control circuitry configured to: analyze a subject image to detect a first person and a second person depicted in the subject image; identify at least one of the first person or the second person in at least one stored image of a plurality of stored images, wherein the plurality of stored images is associated with a plurality of persons; determine a relationship between the first person and the second person in the subject image based at least in part on information corresponding to the at least one stored image; associate an indication of the determined relationship with metadata of the subject image; and generate, for simultaneous display with the subject image, a visual identifier that indicates the determined relationship between the first person and second person (See claim 1 for detailed analysis.).
As to claim 27, claim 26 is incorporated and the combination of Ganong and Greenberger discloses the plurality of stored images comprises a profile picture for each of a plurality of profiles of a social media platform; at least one of the first person or the second person of the subject image is identified in at least one profile picture; and the information corresponding to the at least one stored image comprises information from a profile of the plurality of profiles that corresponds to the at least one profile picture (See claim 2 for detailed analysis.).
As to claim 28, claim 27 is incorporated and the combination of Ganong and Greenberger discloses the profile of the plurality of profiles that corresponds to the at least one profile picture is a first profile; the plurality of stored images comprises non-profile pictures for at least a portion of the plurality of profiles of the social media platform; the control circuitry is further configured to:identify at least one of the first person or the second person by (i) identifying the first person in the at least one profile picture and (ii) determining that the profile pictures of the social media platform do not comprise the second person; and based at least in part on the determining that the profile pictures of the social media platform do not comprise the second person, identify the second person in at least one stored image of the non-profile pictures; and wherein the determining the relationship between the first person and the second person in the subject image is further based on information from a second profile of the plurality of profiles that corresponds to the at least one stored image of the non-profile pictures (See claim 3 for detailed analysis.).
29.-31. (Canceled)
As to claim 32, claim 26 is incorporated and the combination of Ganong and Greenberger discloses the control circuitry is further configured to identify at least one of the first person or the second person by identifying the first person and not the second person in the at least one stored image of the plurality of stored images; wherein the information corresponding to the at least one stored image is information associated with the first person; the control circuitry is further configured to determine information about the second person based at least in part on at least one of (i) characteristics of the subject image or (ii) the information associated with the first person; and wherein the determining the relationship between the first person and the second person in the subject image is further based on the determined information about the second person (See claim 7 for detailed analysis.).
33.-34. (Canceled).
As to claim 35, claim 26 is incorporated and the combination of Ganong and Greenberger discloses the plurality of stored images are a first plurality of stored images; the control circuitry is further configured to: determine the second person is not in the first plurality of stored images; and based at least in part on determining the second person is not in the first plurality of stored images, identify the second person in at least one stored image of a second plurality of stored images; and wherein the determining the relationship between the first person and the second person is further based on information corresponding to the at least one stored image of the second plurality of stored images (See claim 10 for detailed analysis.).
36.-37. (Canceled).
As to claim 38, claim 26 is incorporated and the combination of Ganong and Greenberger discloses the relationship between the first person and the second person in the subject image is a first relationship determined at a first time; and the control circuitry is further configured to: determine a second relationship between the first person and the second person based at least in part on information corresponding to the at least one stored image at a second time later than the first time, wherein the information from the second time is different from the information from the first time; and update the metadata of the subject image with an indication of the determined second relationship (See claim 13 for detailed analysis.).
39.-43. (Canceled).
As to claim 45, claim 26 is incorporated and the combination of Ganong and Greenberger discloses the control circuitry is further configured to generate, for simultaneous display with the subject image, the visual identifier by modifying, based at least in part on the determined relationship between the first person and second person, an appearance of at least one of the first or second person depicted in the subject image (See claim 20 for detailed analysis.).
As to claim 46, claim 26 is incorporated and the combination of Ganong and Greenberger discloses the control circuitry is further configured to associate an indication of the determined relationship with metadata of the subject image by modifying the metadata to include any one or more of (i) an indication of one or more portions of the subject image that comprise at least one of the depiction of the first or second person, (ii) an identity of at least one of the first person or the second person, (iii) the information corresponding to the at least one stored image, or (iv) information about a type of the visual identifier. (See claim 21 for detailed analysis.).
As to claim 47, claim 26 is incorporated and the combination of Ganong and Greenberger discloses the system further comprises input/output circuitry configured to receive, from a user device, a request for relationship information between two persons depicted in a particular image; and the control circuitry is further configured to: determine the particular image is the subject image, and the two persons are the first and second person; and provide, to the user device, the metadata without the subject image (See claim 22 for detailed analysis.)..
48.-125. (Canceled).
Claims 19 and 44 are rejected under 35 U.S.C. 103 as being unpatentable over Ganong et al. (US Pub 2018/0046855 A1) in view of Greenberger et al. (US Pub 2018/0101973 A1) and Costin et al. (US Pub 2024/0135611 A1).
As to claim 19, claim 1 is incorporated and the combination of Ganong and Greenberger does not disclose the generating, for simultaneous display with the subject image, the visual identifier comprises modifying, based at least in part on the determined relationship between the first person and second person, the subject image to adjust a location of the first person in relation to the second person within the subject image.
Costin teaches generating, for simultaneous display with the subject image, the visual identifier comprises modifying, based at least in part on the determined relationship between the first person and second person, the subject image to adjust a location of the first person in relation to the second person within the subject image (Costin, ¶0146, “updating a scene graph can include adding a node representing a new object added to the image, or modifying a node by changing an attribute of an object present in the image. Updating a scene graph can also include modifying a node and/or edge by changing the relationship between two or more objects in the image, such that object positions (e.g., location, orientation, pose, etc.) change relative to each other. The modification to the original image can be stored as information in one or more nodes.”).
Ganong, Greenberger and Costin are considered to be analogous art because all pertain to image processing. It would have been obvious before the effective filing date of the claimed invention to have modified Ganong with the features of “generating, for simultaneous display with the subject image, the visual identifier comprises modifying, based at least in part on the determined relationship between the first person and second person, the subject image to adjust a location of the first person in relation to the second person within the subject image” as taught by Greenberger. The suggestion/motivation would have been in order to applying an image overlay to represent relationships between objects (Costin, ¶0145.)
As to claim 44, claim 26 is incorporated and the combination of Ganong, Greenberger and Costin the control circuitry is further configured to generate, for simultaneous display with the subject image, the visual identifier by modifying, based at least in part on the determined relationship between the first person and second person, the subject image to adjust a location of the first person in relation to the second person within the subject image (See claim 19 for detailed analysis.).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Bullock et al. (US Pub 2018/0332140 A1) discloses tag information of each photo indicating relationships (e.g., a tag indicating a husband, wife, daughter, etc.), status updates, text associated with photos or comments under photos indicating household members' relationships.
Fadeev et al. (US Pub 2018/0314880 A1) discloses identifying a user across multiple online systems based on image data.
Li et al. (US Pub 2015/0262037 A1) discloses harnessing the social network of individuals identified from one or more image collections.
Bourdev et al. (US Pub 2015/0036919 A1) discloses image classification by correlating contextual cues.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to YU CHEN whose telephone number is (571)270-7951. The examiner can normally be reached on M-F 8-5 PST Mid-day flex.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Xiao Wu can be reached on 571-272-7761. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/YU CHEN/Primary Examiner, Art Unit 2613
Prosecution Insights