DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 10 is rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
Claim 10 recites that “the at least one processor is further configured to execute the one or more instructions to select the reference image associated with the location information having a predetermined relationship with the related area of the target contributor.” However, the claim and specification fail to define what constitutes a “relationship,” how the relationship is “predetermined,” and whether the relationship is social, geographic, temporal, behavioral, systems-defined, or something else. As written, “predetermined relationship” lacks objective boundaries and leaves the scope of the claim unclear. Different interpretations (e.g., friendship, follower status, shared location history, similarity metrics, or manual configuration) would materially affect claim scope.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The apparatus of claim 1 is directed to a machine, which is one of the statutory categories of invention, and passes Step 1: Statutory Category- MPEP § 2106.03. However, the following limitations of Claim 1 recite steps that can be performed in the human mind or with pen and paper, therefore failing Step 2A Prong One. These limitations constitute mental processes because they describe acts of observation, evaluation, and judgement that can practically be performed in the human mind, or by a human using pen and paper as a physical aid.
determine a related area of a target contributor who posted the posted image to be processed;
select a reference image to be collated with the posted image to be processed, based on a determination result of the related area of the target contributor;
and estimate a location indicated by the posted image to be processed, based on a collation result between the selected reference image and the posted image to be processed.
Claim 1 fails Step 2A Prong Two because the additional elements beyond the judicial exception, including a processor and memory that acquires a posted image to be processed being posted on the Internet, do not integrate the judicial exception into a practical application. Receiving an image is insignificant pre-solution activity (MPEP 2106.05(g)), and there are no improvements to the functioning of a computer or any other technology or technical field (MPEP § 2106.05(a)) as it merely applies the abstract idea on a computer (MPEP § 2106.05(f)). Furthermore, the claim does not impose meaningful limits on the computer components such that the apparatus is tied to a particular machine; the additional elements are described at a high level of generality and can be implemented on any generic computing system (MPEP § 2106.05(b)). Claim 1 also fails Step 2B, as these additional elements are well-understood, routine, and conventional (WURC), adding nothing significantly more than the abstract idea itself (MPEP § 2106.07(a)((III)); a processor, memory, and receiving an image are WURC (see MPEP § 2106.05(d)). As claim 12 and 13 contain this identical ineligible subject matter, they are also rejected.
Claims 2-11 recite steps that can be performed in the human mind or with pen and paper, therefore failing Step 2A Prong One. These limitations constitute mental processes because they describe acts of observation, evaluation, and judgement that can practically be performed in the human mind, or by a human using pen and paper as a physical aid. The only additional element beyond the judicial exception is a processor which fails Step 2A Prong Two and Step 2B (see claim 1 analysis above). As claims 14-20 contain this identical ineligible subject matter, they are also rejected.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1-3, 5, 11-15, and 17-19 are rejected under 35 U.S.C. 102 (a)(1) as being anticipated by Flynn et. al (US 8,189,964 В2).
Regarding Claim 1, Flynn teaches an estimation apparatus comprising:
Summary of the Invention, lines 31-32: “In another embodiment, an image processing apparatus is provided.”
at least one memory configured to store one or more instructions (Fig. 7 (shown below));
PNG
media_image1.png
603
917
media_image1.png
Greyscale
and at least one processor configured to execute the one or more instructions to (Fig. 7 (shown above));
acquire a posted image to be processed being posted on the Internet;
Summary of Invention, lines 32-34: “The method comprises receiving an image request from a user device, the image request including an image of interest and location metadata for the image of interest.”
Column 7, lines 20-24: “The front-end server 502 receives an image/match request, for instance from an application or interface on the mobile user device. The request includes an image and corresponding metadata about the image's geographical location and orientation (if available).”
determine a related area of a target contributor who posted the posted image to be processed;
Summary of Invention, lines 34-36: “Analyzing the location metadata to select one or more cells to evaluate against the image of interest, each cell having one or more geolocated images and index data associated therewith…”
Column 7, lines 24-27: “The front-end server 502 uses the image's received location information, plus an estimate of any possible error in the location information, to determine a small subset of cells to match the image against.”
Explanation: Cells are explicitly geographic regions, showing a related area determination.
select a reference image to be collated with the posted image to be processed, based on a determination result of the related area of the target contributor;
Column 2, lines 2-4: “Each cell has one or more geolocated images and index data associated therewith…”
Column 6, lines 64-67: “A received image may be quickly matched against a given index. By way of example only, the indices may be written to a key-value store database, where the key is the cell's unique ID.”
Column 7, lines 35-36: “Each cell match server 504 matches the received image against its respective index data.”
Summary of the Invention, lines 38-41: “Comparing the image of interest against the index data of that cell; identifying any matches from the geolocated images of the selected cells based on the compared index data; and providing the matches.”
Explanation: Selection of which images are evaluated is driven by the cell selection, which directly shows selecting reference images based on the area. It also shows image collation and matching.
and estimate a location indicated by the posted image to be processed, based on a collation result between the selected reference image and the posted image to be processed.
Column 6, lines 16-18: “Once the received image is matched to a known image, a more refined location can be associated with the received image.”
Summary of the Invention, lines 48-50: “In another alternative, updated location metadata for the image of interest is provided to the user device along with the matches.”
Regarding Claim 2, Flynn teaches the estimation apparatus according to claim 1, wherein the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on location information included in account information of the target contributor (Column 7, lines 20-24 (shown above)).
Regarding Claim 3, Flynn teaches the estimation apparatus according to claim 1, wherein the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on at least one of a subject of the posted image to be processed, and a subject of a posted image of the target contributor in past.
Column 4, lines 34-36: “In order to overcome such problems, one aspect of the invention matches the photograph against a database of reference images.”
Column 2, lines 22-26: “In a further alternative, the index data corresponds to features of the geolocated images. And in this case, the features are desirably selected from the set consisting of corners, edges or lines, brightness information and histogram information.”
Explanation: Subjects and features of images are explicitly disclosed.
Regarding Claim 5, Flynn teaches the estimation apparatus according to claim 1, wherein the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on location information associated with a posted image of the target contributor in past.
Abstract: “The image of interest may include some location information itself, such as latitude/longitude coordinates and orientation.”
Explanation: Database contains stored historical geolocated images.
Regarding Claim 11, Flynn teaches the estimation apparatus according to claim 1, wherein the reference image is associated with location information, and the at least one processor is further configured to execute the one or more instructions to select the reference image associated with the location information having a predetermined relationship with the related area of the target contributor (Abstract (shown above)).
Regarding Claim 12, Flynn teaches all of the limitations of claim 1 above because claim 12 recites a method that performs substantially the same steps as claim 1.
Regarding Claim 13, Flynn teaches all of the limitations of claim 1 above because claim 13 recites a non-transitory storage medium that performs substantially the same steps as claim 1.
Regarding Claim 14, Flynn teaches the estimation method according to claim 12, and additional limitations are met as in the consideration of claim 2 above.
Regarding Claim 15, Flynn teaches the estimation method according to claim 12, and additional limitations are met as in the consideration of claim 3 above.
Regarding Claim 17, Flynn teaches the estimation method according to claim 12, and additional limitations are met as in the consideration of claim 5 above.
Regarding Claim 18, Flynn teaches the non-transitory storage medium according to claim 13, and additional limitations are met as in the consideration of claim 2 above.
Regarding Claim 19, Flynn teaches the non-transitory storage medium according to claim 13, and additional limitations are met as in the consideration of claim 3 above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 4, 16, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Flynn et. al in view of McMahon and Mont-Reynaud (US 20140078331 A1).
Regarding Claim 4, Flynn teaches the estimation apparatus according to claim 1, but fails to teach that the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on at least one of a sound related to the posted image to be processed, and a sound related to a posted image of the target contributor in past.
However, McMahon and Mont-Reynaud teach capturing and associating sound data with captured images and processing such sound data to generate identifying metadata associated with the image, stating that “the image capture device is configured to capture a sound while capturing an image” (paragraph [0006]). McMahon and Mont-Reynaud further teach processing the captured sound to generate identification data and associating that data with the image, stating that “the captured sound is processed to generate sound identification data…the sound identification data is associated with the image” (Abstract). McMahon and Mont-Reynaud also teach that such sound-derived data may include contextual metadata useful for searching or identifying image content, stating that “the sound-derived identification data may include a transcription for speech, or audio or music meta data…once the association has been stored in different ways for different purposes, searching by one field or another becomes possible…user name, time or geo tag, music meta data and even image content may all serve as the basis for specialized search interfaces” (paragraph [0005] and [0017]).
Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Flynn to incorporate sound-derived metadata as taught by McMahon and Mont-Reynaud in order to improve contextual identification and classification of images. McMahon and Mont-Reynaud teach that combining audio data with visual data enhances identification and retrieval of content, stating that “audio identification and image recording technologies exist separately, and users cannot capture and identify a momentary audio experience along with simultaneous visual material…thus, there exists a need for identifying and interacting jointly with visual and audio data” (paragraph [0004]). Incorporating McMahon and Mont-Reynaud’s sound identification techniques would have predictably enhanced Flynn’s system by providing additional contextual data associated with contributor content, improving accuracy and robustness of determining the related area of the contributor.
Regarding Claim 16, Flynn teaches the estimation method according to claim 12, and additional limitations are met as in the consideration of claim 4 above.
Regarding Claim 20, Flynn teaches the non-transitory storage medium according to claim 13, and additional limitations are met as in the consideration of claim 4 above.
Claims 6-10 are rejected under 35 U.S.C. 103 as being unpatentable over Flynn et. al in view of Barrington et. al (US 9,032,000 В2).
Regarding Claim 6, Flynn teaches the estimation apparatus according to claim 1, but fails to teach that the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on at least one of a keyword included in a posted text being posted together with the posted image to be processed, and a keyword included in a posted text of the target contributor in past.
However, Barrington discloses that “metadata storage may be used to house a repository of known social media content metadata such as usernames, social media identifiers such as "hash tags" used to categorize and qualify posted content, keywords or other metadata” (Column 2, lines 1-3), thus extracting keywords from text associated with a current posting. Barrington further discloses determining location by “comparison of current content to previously analyzed content for analysis (such as referring to previous content posted by a user where a location may have been known)” (Column 2, lines 29-32), which involves keywords from past posts of the same contributor. In addition, Barrington teaches “identifying any known place names present in or associated with the social media content posting” (Column 3, lines 39-41), thus using extracted keywords to infer geographic location.
Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Flynn to determine the related area using keywords from present and past contributor text as taught by Barrington because Barrington identifies keyword-based textual analysis as a reliable signal for determining contributor location which would improve image-matching accuracy when visual cues alone are insufficient.
Regarding Claim 7, Flynn teaches the estimation apparatus according to claim 1, but fails to teach that the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on a language used by the target contributor.
However, Barrington discloses that “content may be scanned for location names which may be used as location cues” (Column 11, lines 66-67) and that “content's text may be searched for additional cues regarding context” (Column 12, lines 2-3), which involves analyzing the language used in the posted text. Barrington further discloses “identifying an author of the social media content posting” (Column 3, lines 34-35) and performing “comparison of current content to previously analyzed content for analysis (such as referring to previous content posted by a user where a location may have been known)” (Column 2, lines 29-32), thus associating the analyzed language with the specific contributor.
Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate language-based contributor analysis into Flynn because Barrington teaches that linguistic context derived from contributor-authored content provides meaningful geographic signals that improve location determination of social media posts.
Regarding Claim 8, Flynn teaches the estimation apparatus according to claim 1, but fails to teach that the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on post timing of the target contributor.
However, Barrington discloses that metadata “may include usernames, proper names, posting date or time…” (Column 11, lines 51-52), thus extracting temporal information associated with a contributor’s past. Barrington further teaches determining location by “determining if location data pertaining to a physical location of the author at the time of posting of the social media content posting is available…” (Column 12, lines 58-60), which directly correlates posting time with contributor location.
Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to modify Flynn to use post timing as an additional factor in determining related area because Barrington teaches that posting time correlates with contributor location and time zone which would improve the accuracy of geolocation estimation.
Regarding Claim 9, Flynn teaches the estimation apparatus according to claim 1, but fails to teach that the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on an activity area of the target contributor.
However, Barrington discloses “searching a user's previous posts to determine if a location was ever revealed” (Column 11, lines 13-14), thus identifying geographic areas associated with the contributor’s prior activity. Barrington further teaches “comparison of current content to previously analyzed content for analysis (such as referring to previous content posted by a user where a location may have been known)” (Column 2, lines 29-32), which uses historical activity patterns to infer location. Additionally, Barrington discloses that “relevancy scores may be attributed to various data and results of processing steps…such as the author's last known location” (Column 12, lines 17-26), indicating that repeated posting behavior and prior locations define a contributor’s geographic activity area.
Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate contributor activity-area analysis into Flynn because Barrington teaches that aggregating prior contributor activity provides a reliable geographic footprint that enhances current post geolocation accuracy.
Regarding Claim 10, Flynn teaches the estimation apparatus according to claim 1, but fails to teach that the at least one processor is further configured to execute the one or more instructions to determine the related area of the target contributor, based on the related area of a related contributor having a predetermined relationship with the target contributor.
However, Barrington discloses “identifying an author of the social media content posting” (Column 3, lines 34-35) and “determining if the metadata storage contains location-related data about the author” (Abstract), thereby establishing contributor-to-contributor relationships via stored author identifiers and associated metadata. Barrington further explains that location determination is performed by “comparison of current content to previously analyzed content for analysis (such as referring to previous content posted by a user where a location may have been known)” (Column 2, lines 29-32), which relies on location information derived from content posted by a related contributor. Additionally, Barrington discloses that “metadata storage may be used to house a repository of known social media content metadata such as usernames, social media identifiers such as "hash tags" used to categorize and qualify posted content, keywords or other metadata” (Column 2, lines 1-3), thus defining predetermined relationships between contributors whose content is stored and associated with known locations.
Thus, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate related-contributor location information into Flynn’s system as taught by Barrington because Barrington recognizes that social relationships provide additional contextual signals useful for resolving geographic ambiguity of social media content.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Nielsen (US 20160117347 A1) teaches constructing a social media identity by analyzing images, metadata, and geolocation information from a user’s social media posts and those of the user’s social connections. The reference uses image recognition and clustering of geolocation data associated with related contributors to infer user identity, relationships, and geographic patterns.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to WILLIAM ADU-JAMFI whose telephone number is (571)272-9298. The examiner can normally be reached M-T 8:00-6:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Andrew Bee can be reached at (571) 270-5183. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/WILLIAM ADU-JAMFI/Examiner, Art Unit 2677
/ANDREW W BEE/Supervisory Patent Examiner, Art Unit 2677