DETAILED ACTION
Notice of Pre-AIA or AIA Status.
1. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
2. Claims 2-21 filed and preliminary amended on 06/20/2025 are pending and being examined. Claims 2 and 12 are independent form.
Priority
3. This application is a CON of 17/106,900 filed on 11/30/2020, now PAT 12112572, 17/106,900 is a CON of 15/924,928 filed on 03/19/2018, now PAT 10885315.
Claim Rejections - 35 USC § 103
4. In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
5. The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
6. Claims 2-21 are rejected under 35 U.S.C. 103 as being unpatentable over Henne et al (“SnapMe if You Can: Privacy Threats of Other Peoples’ Geo-tagged Media and What We Can Do About It”, ACM, 2013 hereinafter “Henne”) in view of Zerr et al (“Privacy-Aware Image Classification and Search”, 2012, hereinafter “Zerr”).
Regarding claim 2, Henne discloses a computer-implemented method (the SnapMe privacy watchdog service/method; see figs. 6-7 and Sec. 6), comprising:
receiving a request for a first action to be performed with respect to a first media asset of a plurality of media assets, wherein the first action comprises deleting one or more segments of the first media asset depicting a user in a first depiction type (wherein the user 2 has requested for protecting his/her privacy information including the location information and face information depicted his/her photo taken by the user 1 cameraman; see section 6.1.1: “SnapMe checks if the location has been marked as private by a user, [,].” It should be noticed that the first media asset and first action are interpreted as the user 2’s photo and protecting the user 2’s privacy information, respectively);
causing the first action to be performed with respect to the one or more segments of the first media asset; determining, are different from the first media asset; based on the scanning, identifying a second media asset from the plurality of media assets that depicts the user; calculating a level of similarity based on comparing the depiction of the user in one or more segments of the second media asset with the first depiction type; upon determining that the calculated level of similarity between the depiction of the user in the one or more segments of the second media asset and the first depiction type is above a particular level of similarity, causing the first action to be performed with respect to the one or more segments of the second media asset (to notify the user whether his/her face has showed in the uploaded photo which is taken within the his/her defined “static area”, the SnapMe photo inspection algorithm, shown by fig.7, may (1) “check user’s privacy area” by extracting the location information from the uploaded photo and comparing/calculating the distance between the location extracted from the uploaded photo to the defined static area registered in the database, (2) perform “face recognition” for user’s identification, and the (3) take an action including notifying the user according to the detected location information and the face recognition result; see fig.7. It should be noticed that, “the first depiction type” and ‘the second media asset” are interpreted as the location information extracted from the uploaded photo and the location information—namely, the defined static area, registered in the database).
As explained above, the mere difference is, Henne does not explicitly discloses “using a machine learning (ML) model” to identify textural features and facial features in photo images. However, in the same field of endeavor, Zerr discloses a supervised learning paradigm support vector machine (SVM) which is trained based on training photos for identifying textural features and facial features in photo images. See Sec. 5.1, para.1; see Sec. 4, para.2. It would have been obvious to one of ordinary skill in the art before the effective filling date of the claimed invention was made to incorporate the teachings of Zerr into the teachings of Henne and utilize a support vector machine (SVM) for identifying the textual information and the face information in a photo. Suggestion or motivation for doing so would have been to “detect private images” and “enable privacy-oriented image search” as taught by Zerr, cf., Abstract. Therefore, the claim is unpatentable over Henne in view of Zerr.
Regarding claim 3, 13, the combination of Henne and Zerr discloses, further comprising: upon determining that the calculated level of similarity between the depiction of the user in the one or more segments of the second media asset and the first depiction type is not above the particular level of similarity, determining, based on user profile data, whether the depiction of the user in the one or more segments of the second media asset is undesirable to the user; and based on determining that the depiction of the user in the one or more segments of the second media asset is undesirable to the user, causing the first action to be performed with respect to the one or more segments of the second media asset (Zerr, to classify textural features and facial features in photo images and determine the degrees of each photo image classification, the system in Zerr determines a “similarity” between two information nuggets extracted from photo images for ranking the search results; see Eq(4), Eq(3)).
Regarding claim 4, 14, the combination of Henne and Zerr discloses, further comprising: upon determining that the calculated level of similarity between the depiction of the user in the one or more segments of the second media asset and the first depiction type is not above the particular level of similarity, providing, for display at a user device, a user selectable option to select a second action to be performed with respect to the one or more segments of the second media asset, wherein the second action is different from the first action; based on receiving the selection of the second action to be performed with respect to the second media asset, causing the second action to be performed with respect to the one or more segments of the second media asset; and determining, using the ML model, that the depiction of the user in the one or more segments of the second media asset is a second depiction type that is undesirable to the user, wherein the second depiction type is different from the first depiction type (Henne, the system may provide a user selectable option, see Sec. 1.1.1, para.1: “SnapMe checks if the location has been marked as private by a user, or if a user was at the time and place where the picture was taken. The location information (GPS coordinates or textual information) of a photo can be embedded in its metadata or sent as context data by the uploading client application. In the latter case, users might select if location information should only be used by the watchdog and thus not be permanently stored afterwards, or if it should be published with the image.”).
Regarding claim 5, 15, the combination of Henne and Zerr discloses, wherein a data structure of a plurality of data structures is generated for each respective media asset of the plurality of media assets; and wherein each data structure comprises an identifier identifying the respective media asset, an indication as to whether a depiction type associated with the respective media asset is undesirable to the user, and one or more keywords associated with the media asset (Henne, wherein the “metadata contained information and potentially derivable private information” includes a user’s identifier-- the name of a user; see the third box of fig.1).
Regarding claim 6, 16, the combination of Henne and Zerr discloses, wherein the determining, using the ML model, that the first depiction type is undesirable to the user further comprises: adding to a first data structure corresponding to the first media asset an indication that the first depiction type is undesirable to the user; and causing the first action to be performed with respect to the second media asset based at least in part on the indication in the first data structure that the first depiction type is undesirable to the user (Henne, creating a metadata corresponding to each of the users , wherein metadata includes an user’s private information to be remove, which may be structed by the user’s name, the keywords, the user’s photo’s textual description, and the pgs location as shown by fig.1).
Regarding claim 7, 17, the combination of Henne and Zerr discloses, further comprising: based at least in part on the first depiction type, generating, by the ML model, a first one or more keywords; storing the first one or more keywords in a first data structure of the plurality of data structures corresponding to the first media asset; generating, based on the depiction of the user in the one or more segments of the second media asset, a second one of more keywords; and storing the second one or more keywords in a second data structure of the plurality of data structures corresponding to the second media asset (Henne, creating a metadata corresponding to each of the users , wherein metadata includes an user’s private information, which may be structed by the user’s name, the “keywords”, the user’s photo’s textual description, and the pgs location as shown by fig.1).
Regarding claim 8, 18, the combination of Henne and Zerr discloses, wherein the calculating a level of similarity is based on: comparing the first data structure and the second data structure; and based on the comparing, determining that at least a portion of the first one or more keywords matches or is synonymous with at least a portion of the second one or more keywords (Henne, e.g., the location/keywords matching, see fig.7 and Sec. 6.1.1, the past para: “SnapMe searches for users in the photo’s vicinity and users that have set up matching static areas.”).
Regarding claim 9, 19, the combination of Henne and Zerr discloses, wherein the determining, using the ML model, that the first depiction type is undesirable to the user is further based on: extracting, by the ML model using an image analysis algorithm, the first depiction type from the first media asset; identifying user profile data associated with the user; and determining whether the extracted first depiction type is undesirable to the user based on user profile data (see Sec. 6.1.2, para.2: [to perform face recognition,] “[t]he training data for the face recognition algorithm is of course a very interesting issue: In many cases SNS already have an extensive library of profile pictures that are already in use for face recognition.”).
Regarding claim 10, 20, the combination of Henne and Zerr discloses, wherein the calculating a level of similarity is further based on using, by the ML model, a K-nearest neighbor classifier algorithm to compare the first depiction type with the depiction of the user in the one or more segments of the second media asset (Zerr, to classify textural features and facial features in photo images and determine the degrees of each photo image classification, the system in Zerr determines a “similarity” between two information nuggets extracted from photo images for ranking the search results; see Eq(4), Eq(3)).
Regarding claim 11, 21, the combination of Henne and Zerr discloses, wherein the identifying the second media asset from the plurality of media assets that depicts the user is based on: identifying an image signature corresponding to the user from a stored plurality of image signatures, wherein each image signature of the stored plurality of image signatures corresponds to a respective user of a plurality of users; and calculating a similarity between the identified image signature and the second media asset depiction of the user (Zerr, to classify textural features and facial features extracted from photo images and determine the degrees of each photo image classification, the system in Zerr determines a “similarity” between two information nuggets extracted from photo images for ranking the search results; see Eq(4), Eq(3)).
Regarding claim 12, claim 12 is an inherent variation of claim 2, thus it is interpreted and rejected for the reasons set forth in the rejection of claim 2.
Conclusion
7. Any inquiry concerning this communication or earlier communications from the examiner should be directed to RUIPING LI whose telephone number is (571)270-3376. The examiner can normally be reached 8:30am--5:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, HENOK SHIFERAW can be reached on (571)272-4637. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit https://patentcenter.uspto.gov; https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center, and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RUIPING LI/Primary Examiner, Ph.D., Art Unit 2676
Prosecution Insights