Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claims 1-20 are rejected under 35 U.S.C. 112(b) or 35 U.S.C. 112 (pre-AIA ), second paragraph, as being indefinite for failing to particularly point out and distinctly claim the subject matter which the inventor or a joint inventor (or for applications subject to pre-AIA 35 U.S.C. 112, the applicant), regards as the invention.
With regard to claim 1, the claim recites “determining candidate documents including documents with images semantically similar to the query image from an image index and documents responsive to the context query from a document index;” This claim limitation lacks antecedent basis.
Each unique claim label is expected to refer to a unique claim element, and each unique claim element is expected to have a unique claim label. The use of the same label “documents” to refer to two distinct elements (e.g. 1, semantically similar images from the image index and 2, results from the document index) lacks antecedent basis. It is unclear if applicant is intending to recite two distinct documents within the candidate documents, or if applicant is attempting to recite a single document that satisfies both criteria.
This issue is further complicated by claim 2 which recites “wherein the candidate documents are first candidate documents”. As such, adding simple label distinctions of ‘first’ and ‘second’ to the documents would introduce confusion within claim 2 regarding the ‘first candidate documents’ and ‘second candidate documents’.
For examination purposes this claim limitation has been construed to mean -- determining candidate documents including image documents with images semantically similar to the query image from an image index and context documents responsive to the context query from a document index --. Furthermore, this claim limitation has been construed in light of Paragraph [0035] of the original specification which describes the “image index 134” and “document index 130” as being the same posting list structure where the images are associated with terms.
With regard to claims 2 and 18, claim 2 recites “determining, for the documents with images semantically similar to the query image”…
filtering out documents from the second candidate documents in response to determining the documents lack an image that meets a visual similarity threshold with the query image;”.
This claim limitation lacks antecedent basis. Claim 18 is rejected based upon similar reasoning and rational as claim 2 as it appears to recite substantially similar language. The parent claim appears to recite two distinct elements labeled ‘documents’ and it is unclear which ‘documents’ is being referenced to.
For examination purposes this claim limitation has been construed to mean --determining, for the image documents with images semantically similar to the query image … filtering out dissimilar documents from the second candidate documents in response to determining the dissimilar documents lack an image that meets a visual similarity threshold with the query image; --.
With regard to claims 2 and 18, claim 2 recites “including the second candidate documents with the first candidate documents as part of determining highest-ranking documents.” This claim limitation lacks antecedent basis. Claim 18 is rejected based upon similar reasoning and rational as claim 2 as it appears to recite substantially similar language. The parent claim does not recite determining highest-ranking documents. The parent claim at best recites “ranking the candidate documents based on similarity to the context query to generate highest ranking candidate documents”. It is noted that this language of the parent claim does not actually recite that the claimed method performs the generation of the highest ranking candidate documents, but instead recites this as an intended use of the ranking operation.
To be clear, the instant claim limitations appear to refer to a method step that is not previously recited for two reasons. First, the parent claim does not recite “determining highest-ranking documents”. Second, even if one of ordinary skill identified this as referring to the recited “generating highest ranking candidate documents”, this is not an operation required to be performed by the claimed method as it is recited as an intended result of the “ranking the candidate document” step.
It is suggested that the claims be amended to use consistent language so that it is clear what is being referenced, and that the claims be amended to recite the desired method steps in a positive manner.
For examination purposes this claim limitation has been construed to mean -- including the second candidate documents with the first candidate documents when ranking the first candidate documents--.
With regard to claims 3 and 19, claim 3 recites “wherein the candidate documents further include documents from a fact-check repository that include an image similar to the query image.” This claim lacks antecedent basis. Claim 19 appears to recite substantially similar language and is rejected based upon similar reasoning and rational. For examination purposes this claim limitation has been construed to mean -- -- wherein the candidate documents further include the image documents from a fact-check repository that include a stored image similar to the query image.—
With regard to claim 7, this claim limitation recites “receiving a search result from the image context service in response to providing the image, the search result being based on a ranking of documents that have images semantically or visually similar to the image and a relevance to a context query related to the image;”. This claim lacks antecedent basis. It is unclear if/when applicant is attempting to recite a new claim element or refer to the previously recited ‘image’. For examination purposes it is suggested that the claims be amended to give each claim element a clearly unique label, and to use these labels consistently.
For examination purposes this claim limitation has been construed to mean -- receiving a search result from the image context service in response to providing the image, the search result being based on a ranking of documents that have document images semantically or visually similar to the image and a relevance to a context query related to the image;--.
With regard to claim 8, the claim recites “wherein a source document associated with the image is provided…” This claim limitation lacks antecedent basis as it is unclear if applicant is attempting to refer to the “receiving a request for context about an image” or the “documents that have images semantically or visually similar”.
Please note the 112b rejection for claim 7, as the correction for that issue may impact the interpretation of the language in this claim. It is suggested that claims 7 and 8 be amended to recite the ‘image’ as --the query image-- so that it is clearly distinct from the --document image-- found in the result documents. This would further allow the claims to be amended to clearly articulate which ‘image’ is being referenced to (e.g. query or result document image).
For examination purposes this claim limitation has been construed as referring to the --query image—(e.g. ‘the image’ recited in claim 7 in the language “receiving a request for context about an image”).
With regard to claim 12, the claim recites “generating a context query for a query image from terms describing the query image that are obtained from a generative model provided the query image as input;” This claim limitation lacks antecedent basis. It is unclear what is obtained from the generative model. One of ordinary skill in the art may reasonably read this claim limitation as referring to the context query, the query image, or the terms. For examination purposes this claim limitation has been construed to mean -- generating a context query for a query image from terms describing the query image, wherein the context query is obtained from a generative model provided the query image as input;--.
With regard to claim 17, the claim recites “wherein determining the candidate documents further includes identifying documents with images semantically similar to the query image from an image index.” This claim lacks antecedent basis, as the claim has previously recited ‘documents’ within a different context. For examination purposes this claim limitation has been construed to mean -- wherein determining the candidate documents further includes identifying similar documents with images semantically similar to the query image from an image index.--.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 4-8, 10-17, and 20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Caid [6173275]. Please note that Caid [6173275] (herein referred to as Caid275) incorporates Caid [5619709] (herein referred to as Caid709) by reference. Caid275 makes frequent reference to Caid709 when describing the disclosed device. Within this office action, mapping citations for the disclosed device will be provided directly to Caid275 and Caid709.
With regard to claim 1 Caid (Caid275 incorporating Caid709) teaches A method comprising:
generating a context query (Caid275, Column 13, line 10-11 “user queries 1301 are converted into a query context vector 1303.”) for a query image (Caid275, Column 12, line 51 “Image Queries”) based on a source document as a magazine (Caid275, Column 14, lines 41-42 “Similarly, text and images can be linked, for example magazines and pictures in magazines”) associated with the query image as the image being searched (Caid275, Column 12, line 51 “Image Queries”);
determining candidate documents including documents as results 1307 (Caid275, Column 13, lines 15-18 “Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307. The images are sorted by dot product 1309.”; Please see the 112b above, this claim limitation has been construed to mean --image document--) with images as retrieved images (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”) semantically similar as dot product similarity (Id; Column 4, lines 49 “wherein a high dot product indicates a similar meaning.”) to the query image as the query context vector represents the query image (Caid275, column 13, lines 10-17 “Regardless of the approach used, user queries 1301 are converted into a query context vector 1303. For example, images and image portions are represented by the atomic vocabulary vectors 205 and a weighted query vector is formed from the atomic vocabulary vectors (see FIG. 5). Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307”) from an image index (Caid275, lines 8 “Images can also be indexed to terms”) and documents as the list of documents (Caid709; Column 18-19 “The net result is a list of the most relevant documents, independent of language”; Please see the 112b above, this claim limitation has been construed to mean --context document--) responsive to the context query as searching for the context query vector using the common universal meaning space (Caid709 Column 24, lines 10-19 describes the search mechanism; Caid275 Column 14, lines 12-15; lines 29-30 “Similarly, a common universal meaning space can be constructed for images (or video) and sound, for example”) from a document index (Caid709 Column 12, lines 39-41 “Another application of the information retrieval system described above is the automated coding of text documents according to a defined index of terms”);
ranking the candidate documents based on similarity to the context query as ordering the list based on similarity weight (Caid275, Column 12, lines 39-42 “The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”) to generate highest ranking candidate documents as the top ten similar images and associated index terms 1208 (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”; Please note that what the ranking of the candidate documents is intended be used for (e.g. “to generate highest ranking candidate documents”) has been an intended use of the ranking operation); and
providing information (Caid275, Column 13, lines 53-56 “Also, as described in Caid et al. ( and incorporated herein), summary vectors of images can be visually displayed for navigation through the corpus of images to find images that are of interest.”) about the highest ranking candidate documents as the top ten similar images and associated index terms 1208 (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”) and information relating to a first appearance as displaying coded information such as date of publication (Caid709 Column 14, lines 11-13 “In addition, coded information about a document (such as the author or the date of publication) can also be represented .”; Column 31, lines 47-49 “For example, one coordinate axis could be used to display the date of production of each data object.”) of the query image as the search query submitted (Caid275, Column 12, line 51 “Image Queries”) to retrieve the search result being displayed with the coded information (Caid709 Column 14, lines 11-13; Column 31, lines 47-49; Please note this claim limitation has been construed in light of Paragraph [0089] of the original specification as being the earliest entry of the identified search result).
With regard to claims 4 and 20 Caid further teaches wherein the candidate documents have respective first relevance scores as similarity 1029 weight (Caid275, Column 12, lines 39-42 “The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”) and the method further comprises:
determining, for the candidate documents as the result documents 1307 (Caid275, Column 13 lines 11-17 “For example, images and image portions are represented by the atomic vocabulary vectors 205 and a weighted query vector is formed from the atomic vocabulary vectors (see FIG. 5). Image relevance is assessed by computing the dot product 15 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307”), respective visual as visual features of identification (Caid275, Column 13, lines 47-51 “The block 1401 with the highest dot product has the highest degree of correlation with the query 1303, and is indicated by some visual distinction such as color, inverse video, or some other form of identification 1407.”) similarity scores as the dot product calculation (Id) based on the query image as the highest degree of correlation with the query (Id) which is determined by a threshold evaluation as a threshold relevance (Caid275, Column 12, lines 37-15); and
boosting as giving a higher weight (Caid275, Column 8, lines 48-51 “The intra-image influence between atoms 511 is distance-weighted according to a Gaussian distribution in that atoms that appear close together in an image are given higher weight.”) the respective first relevance scores as similarity 1029 weight (Caid275, Column 12, lines 39-42 “The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”) based on the respective visual similarity scores as the distance, e.g. the similarity score calculation (Id),
wherein the highest ranking candidate documents as the top ten similar images includes the first and second best matched images (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”) are further ranked as being ordered by vector proximity (Caid275, Column 3, lines 12-14 “Retrieved images are displayed in order of vector proximity, which corresponds to relative relevance to the query”) based on respective second relevance scores as the similarity score (Caid275, Column 12, lines 39-42 “The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”) for a second document in the top ten results (Caid275, Column 12, lines 37-39) determined based on similarity with the context query and the respective first relevance scores as similarity score (Caid275, Column 12, lines 39-42) for a first document in the top ten results (Caid275, Column 12, lines 37-39).
With regard to claim 5 Caid further teaches wherein the context query (Caid275, Column 13, line 10-11 “user queries 1301 are converted into a query context vector 1303.”) is generated from terms most relevant to the query image (Caid275, Column 12, lines 7-9 “Images can also be indexed to terms. Index terms can be textual words or codes, for example. More than one index term can be associated with an image”).
With regard to claim 6 Caid further teaches wherein the candidate documents are filtered as the unretained images, e.g. dissimilar images (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”; Column 2, lines 17-20 “Thus, two items having similar meaning or content have similarly-oriented context vectors, while items having dissimilar meaning or content have roughly orthogonal context vectors” using a document quality threshold as the threshold of best matches (Caid275, Column 12, lines 37-42 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208. The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”) prior as the ordered list is produced from the identified most relevant matched images and terms (Id) to ranking as ordering the list based on similarity weight (Caid275, Column 12, lines 39-42 “The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”).
With regard to claim 7 Caid teaches A method comprising:
receiving a request for context about an image as receiving a query image (Caid275, Column 17, line 32 “receiving a query image”; Caid275, Column 12, line 51 “Image Queries”; Please see the 112b rejection regarding claim 8, this limitation is identified as --query image--);
providing the image (Caid275, Column 13, line 10-11 “user queries 1301 are converted into a query context vector 1303.”; Please see the 112b rejection regarding claim 8, this limitation is identified as --query image--) to an image context service (Figure 1, 114 “Context Vector Generation”; Column 4, lines 4-16 detail some of the underlying hardware components);
receiving a search result as results 1307 (Caid275, Column 13, lines 15-18 “Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307. The images are sorted by dot product 1309.”) from the image context service ((Figure 1, 114 “Context Vector Generation”; Column 4, lines 4-16) in response to providing the image (Caid275, Column 13, line 10-11 “user queries 1301 are converted into a query context vector 1303.”; Please see the 112b rejection regarding claim 8, this limitation is identified as --query image--), the search result being based on a ranking as ordering the list based on similarity weight (Caid275, Column 12, lines 39-42 “The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”) of documents as a magazine (Caid275, Column 14, lines 41-42 “Similarly, text and images can be linked, for example magazines and pictures in magazines”) that have images as retrieved images (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”; Please note the 112b above, this claim limitation has been identified as --document images--) semantically (Caid709 “Column 16, lines 60-61 “a strong similarity of usage or "meaning" between these elements.”) or visually as visual features of identification (Caid275, Column 13, lines 47-51 “The block 1401 with the highest dot product has the highest degree of correlation with the query 1303, and is indicated by some visual distinction such as color, inverse video, or some other form of identification 1407.”) similar as the dot product calculation (Id) to the image as the highest degree of correlation with the query (Id) which is determined by a threshold evaluation as a threshold relevance (Caid275, Column 12, lines 37-15) and a relevance (Caid275 Column 3, lines 12-16 “Retrieved images are displayed in order of vector proximity, which corresponds to relative relevance to the query. In one embodiment, retrieved images are broken into sub-portions and the most relevant portions matching the query vector are highlighted in the image”) to a context query related (Caid275, Column 13, line 10-11 “user queries 1301 are converted into a query context vector 1303.”) to the image (Caid275, Column 12, line 51 “Image Queries”); and
displaying the image (Caid275, Column 3, lines 12 “Retrieved images are displayed”) and the search result as results 1307 (Caid275, Column 13, lines 15-18 “Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307. The images are sorted by dot product 1309.”).
With regard to claim 8 Caid further teaches wherein a source document as a magazine (Caid275, Column 14, lines 41-42 “Similarly, text and images can be linked, for example magazines and pictures in magazines”) associated with the image as the image being searched (Caid275, Column 12, line 51 “Image Queries”; Please see the 112b rejection above, this limitation is identified as --query image--) is provided as receiving a new image (Caid275, Column 15, line 29 “receiving a new image in computer-readable form”) to the image context service (Caid275, Column 12, lines 29-33 “new image 1204 is characterized 1205 using the existing atomic vocabulary 205 and stop list 207 as described above (i.e., wavelet transformation and mapping to closest atoms in atomic vocabulary), and a summary vector 1206 for the image is generated”) and the source document as a magazine (Caid275, Column 14, lines 41-42 “Similarly, text and images can be linked, for example magazines and pictures in magazines”) includes salient terms as text that is deemed important (Caid275, Column 14, lines 41-49 “Similarly, text and images can be linked, for example magazines and pictures in magazines. Here, the text surrounding the image (e.g., in a caption) can be automatically "tied" or associated with the image. The strength of the tie association can also be varied according to spatial proximity of the text to the image, boldness, font, or other factors. See Caid et al. for an application of context vectors to text. Information elements in the textual application may be, for example, words or word stems (root words).”) that are used to generate the context query as being tied to the image (Column 14, lines 29-34 “Similarly, a common universal meaning space can be constructed for images (or video) and sound, for example. Certain "ties" can be fixed between images (e.g., a St. Bernard dog) and audio clips ("bark") ( context vectors in the audio domain may be, for example, frequency and/or amplitude measurements).”).
With regard to claim 10 Caid further teaches wherein receiving the request as the search being a new search that is submitted after the user reviews and specifies previous result images that were useful (Caid275, Column 13, lines 35-40 “The system may also employ relevance feedback, whereby the user specifies which of the retrieved images are most helpful. Anew search may then be performed using the summary vector for the specified images. This technique reduces the time required for searches and improves system effectiveness”) occurs responsive to selection as a double click (Caid709 Column 15, lines 50-53 “Responsive to some command, such as a double-click, associated with one of the icons or sub-icons, the associated document (or portion of a document) may be displayed for perusal by the user.”) of an interactive control as an icon (Id) provided on an image search result page that includes the image as the document being displayed to the user for perusal (Id).
With regard to claim 11 Caid further teaches wherein receiving the request as the search being a new search that is submitted after the user reviews and specifies previous result images that were useful (Caid275, Column 13, lines 35-40 “The system may also employ relevance feedback, whereby the user specifies which of the retrieved images are most helpful. Anew search may then be performed using the summary vector for the specified images. This technique reduces the time required for searches and improves system effectiveness”) occurs responsive to selection as a double click (Caid709 Column 15, lines 50-53 “Responsive to some command, such as a double-click, associated with one of the icons or sub-icons, the associated document (or portion of a document) may be displayed for perusal by the user.”) of an interactive control as an icon (Id) provided on an image search application as the search results page (Id).
With regard to claim 12 Caid teaches A method comprising:
generating a context query (Caid275, Column 13, line 10-11 “user queries 1301 are converted into a query context vector 1303.”) for a query image (Caid275, Column 12, line 51 “Image Queries”) from terms (Caid275, Column 14, lines 41-44 “Similarly, text and images can be linked, for example magazines and pictures in magazines. Here, the text surrounding the image (e.g., in a caption) can be automatically "tied" or associated with the image.”) describing as the terms representing the image (Caid275, Column 7, lines 30-33 “Any image may be represented in terms of the unique set of information elements in the atomic vocabulary 205, 311.”) the query image that are (Caid275, Column 13, line 10-11 “user queries 1301 are converted into a query context vector 1303.”; Please see the 112b above, this claim limitation has been construed to mean –wherein the context query is--) obtained from a generative model (Caid275, Column 6, lines 48 “Statistical prototype feature vectors are selected using neural network self-organization techniques 309”; Figure 1, 114 “Context Vector Generation”; Column 4, lines 4-16) provided the query image as input (Caid275, Column 13, line 10-11 “user queries 1301 are converted into a query context vector 1303.”);
identifying candidate documents as results 1307 (Caid275, Column 13, lines 15-18 “Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307. The images are sorted by dot product 1309.”) responsive to the context query as the query context vector represents the query image (Caid275, column 13, lines 10-17 “Regardless of the approach used, user queries 1301 are converted into a query context vector 1303. For example, images and image portions are represented by the atomic vocabulary vectors 205 and a weighted query vector is formed from the atomic vocabulary vectors (see FIG. 5). Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307”) from a document index (Caid709 Column 12, lines 39-41 “Another application of the information retrieval system described above is the automated coding of text documents according to a defined index of terms”);
ranking the candidate documents based on similarity to the context query as ordering the list based on similarity weight (Caid275, Column 12, lines 39-42 “The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”) to identify highest ranking candidate documents as the top ten similar images and associated index terms 1208 (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”; Please note that what the ranking of the candidate documents is intended be used for (e.g. “to identify highest ranking candidate documents”) has been an intended use of the ranking operation); and
providing information (Caid275, Column 13, lines 53-56 “Also, as described in Caid et al. ( and incorporated herein), summary vectors of images can be visually displayed for navigation through the corpus of images to find images that are of interest.”) about the query image based on the highest ranking candidate documents as the top ten similar images and associated index terms 1208 (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”).
With regard to claim 13 Caid further teaches wherein the terms are first terms (Caid275, Column 14, lines 41-44 “Similarly, text and images can be linked, for example magazines and pictures in magazines. Here, the text surrounding the image (e.g., in a caption) can be automatically "tied" or associated with the image.”), and generating the context query further comprises:
identifying a second image as a second summary vector (Caid275, Column 10 lines 51-53 “Referring to FIG. 10, an initial parent node 1001 at the top of a tree indexed as level 0, node 1, contains all of the summary vectors 913 for all images 201”) that is similar as being in the same cluster as determined by the dot product calculation (Caid275, Column 11, lines 10-14 “Referring to FIG. 10, an initial parent node 1001 at the top of a tree indexed as level 0, node 1, contains all of the summary vectors 913 for all images 201 and recompute the centroid for the cluster which gains a vector 1105;”) to the query image as the query context vector (Caid275, Column 13, lines 21-34 which describes how clusters are used during the search operation), the second image being associated with second terms as the terms tied to the image for the second vector (Caid275, Column 14, lines 41-44 “Similarly, text and images can be linked, for example magazines and pictures in magazines. Here, the text surrounding the image (e.g., in a caption) can be automatically "tied" or associated with the image.”); and
using the second terms in further generating the context query (Column 12, lines 8-12 “Images can also be indexed to terms. Index terms can be textual words or codes, for example. More than one index term can be associated with an image. For example, an image of a dog may be indexed to the textual words "dog", "bark", and "pet".”).
With regard to claim 14 Caid further teaches wherein the second image is visually as visual features of identification (Caid275, Column 13, lines 47-51 “The block 1401 with the highest dot product has the highest degree of correlation with the query 1303, and is indicated by some visual distinction such as color, inverse video, or some other form of identification 1407.”) similar as being in the same cluster as determined by the dot product calculation (Caid275, Column 11, lines 10-14 “Referring to FIG. 10, an initial parent node 1001 at the top of a tree indexed as level 0, node 1, contains all of the summary vectors 913 for all images 201 and recompute the centroid for the cluster which gains a vector 1105”) to the query image as the query context vector (Caid275, Column 13, lines 21-34 which describes how clusters are used during the search operation).
With regard to claim 15 Caid further teaches wherein generating the context query further comprises:
Clustering (Caid275, Column 11, lines 2-4 “In a preferred embodiment, the convergent k-means clustering algorithm, a well-known clustering algorithm, is used.”) the first terms and the second terms as the terms (Column 12, lines 8-12 “Images can also be indexed to terms. Index terms can be textual words or codes, for example. More than one index term can be associated with an image. For example, an image of a dog may be indexed to the textual words "dog", "bark", and "pet".”) of the vectors (Caid275, Column 7 lines 31-32 “Any image may be represented in terms of the unique set of information elements in the atomic vocabulary 205”) to generate a semantic cluster as a centroid cluster (Coid275, Column 10, lines 59-61 “This process can be repeated until a sufficient level of clustering detail is achieved 1007, the result being centroid consistent clusters 1009.”); and
including a description term as the terms of the summary vectors assigned to the parent node of the cluster (Column 11, lines 45-47 “Only the summary vectors assigned to a parent node are used in the clustering algorithm to form a the child nodes which branch from that parent.”) representing the semantic cluster as a centroid cluster (Coid275, Column 10, lines 59-61 “This process can be repeated until a sufficient level of clustering detail is achieved 1007, the result being centroid consistent clusters 1009.”) in the context query as the query context vector (Caid275, Column 13, lines 21-34 which describes how clusters are used during the search operation).
With regard to claim 16 Caid275 further teaches wherein generating the context query further comprises:
determining that a first semantic cluster relates to a first number of as the computed distance between the summary vector and the centroid for a first k-cluster (Caid275, Column 11, lines 14-20 “Take each summary vector in sequence and compute its distance from the centroid of each of the k-clusters 1107 (computed by dot product). If the vector is not currently in the cluster with the closest centroid, move the vector to that cluster and update the centroids of the clusters that gain or lose a summary vector 1109.”) the first terms and the second terms (Column 12, lines 8-12 “Images can also be indexed to terms. Index terms can be textual words or codes, for example. More than one index term can be associated with an image. For example, an image of a dog may be indexed to the textual words "dog", "bark", and "pet".”) and a second semantic cluster relates to a second number as the computed distance between the summary vector and the centroid for a second k-cluster (Caid275, Column 11, lines 14-20 “Take each summary vector in sequence and compute its distance from the centroid of each of the k-clusters 1107 (computed by dot product). If the vector is not currently in the cluster with the closest centroid, move the vector to that cluster and update the centroids of the clusters that gain or lose a summary vector 1109.”) of the first terms and the second terms (Column 12, lines 8-12 “Images can also be indexed to terms. Index terms can be textual words or codes, for example. More than one index term can be associated with an image. For example, an image of a dog may be indexed to the textual words "dog", "bark", and "pet".”), the second number being lower than the first number as if the vector is not currently in the cluster with the closest centroid (Caid275, Column 11, lines 14-20 “Take each summary vector in sequence and compute its distance from the centroid of each of the k-clusters 1107 (computed by dot product). If the vector is not currently in the cluster with the closest centroid, move the vector to that cluster and update the centroids of the clusters that gain or lose a summary vector 1109.”); and
weighting the first semantic cluster higher than the second semantic cluster for inclusion in the context query as moving the vector to the cluster with the shorter distance (Caid275, Column 11, lines 14-20 “Take each summary vector in sequence and compute its distance from the centroid of each of the k-clusters 1107 (computed by dot product). If the vector is not currently in the cluster with the closest centroid, move the vector to that cluster and update the centroids of the clusters that gain or lose a summary vector 1109.”).
With regard to claim 17 Caid275 further teaches wherein determining the candidate documents as results 1307 (Caid275, Column 13, lines 15-18 “Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307. The images are sorted by dot product 1309.”) further includes identifying documents as retrieved images (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”; Please note the 112b above, this claim limitation has been construed to mean --similar documents--) with images (id) semantically similar as dot product similarity (Id; Column 4, lines 49 “wherein a high dot product indicates a similar meaning.”) to the query image as the query context vector represents the query image (Caid275, column 13, lines 10-17 “Regardless of the approach used, user queries 1301 are converted into a query context vector 1303. For example, images and image portions are represented by the atomic vocabulary vectors 205 and a weighted query vector is formed from the atomic vocabulary vectors (see FIG. 5). Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307”) from an image index (Caid275, lines 8 “Images can also be indexed to terms”).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 9 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Caid in view of Mousavian [2020/0363815].
With regard to claims 2 and 18 Caid further teaches wherein the candidate documents are first candidate documents as a first retrieved image (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”) and the method further comprises:
determining, for the documents as a first retrieved image (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”; Please see the 112b above, this claim limitation has been construed to mean --image document--) with images semantically similar as dot product similarity (Id; Column 4, lines 49 “wherein a high dot product indicates a similar meaning.”) to the query image as the query context vector represents the query image (Caid275, column 13, lines 10-17 “Regardless of the approach used, user queries 1301 are converted into a query context vector 1303. For example, images and image portions are represented by the atomic vocabulary vectors 205 and a weighted query vector is formed from the atomic vocabulary vectors (see FIG. 5). Image relevance is assessed by computing the dot product 1305 of each image summary vector 913 with the query context vector 1303, and accumulating the results 1307”),
second candidate documents, the second candidate documents as a second retrieved image (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”) meeting a relevance threshold as a threshold relevance (Caid275, Column 12, lines 37-15 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208. The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213. The new image is then associated with the listed order of index terms 1215. A threshold 1217 can also be used to choose the top 45 N index terms from the weighted list.”) for a [[as the search for magazine pictures, which one of ordinary skill in the art may reasonably recognize as being a ‘stock’ image (Caid275, Column 12, line 51 “Image Queries”; Column 14, lines 41-42 “Similarly, text and images can be linked, for example magazines and pictures in magazines”; Please note this claim limitation has been construed in light of Paragraph [0058] of the original specification as a query that searches for stock photos.);
filtering out documents as the unretained images, e.g. dissimilar images (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”; Column 2, lines 17-20 “Thus, two items having similar meaning or content have similarly-oriented context vectors, while items having dissimilar meaning or content have roughly orthogonal context vectors” ; Please see the 112b above, for examination purposes this claim limitation has been construed to mean –a dissimilar document--) from the second candidate documents as a second retrieved image (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”) in response to determining the documents as the dissimilar images (Caid275 Column 2, lines 17-20; Please see the 112b above, for examination purposes this claim limitation has been construed to mean --dissimilar document--) lack as the unretained images (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”) an image that meets a visual as visual features of identification (Caid275, Column 13, lines 47-51 “The block 1401 with the highest dot product has the highest degree of correlation with the query 1303, and is indicated by some visual distinction such as color, inverse video, or some other form of identification 1407.”) similarity as the dot product calculation (Id) threshold with the query image as the highest degree of correlation with the query (Id) which is determined by a threshold evaluation as a threshold relevance (Caid275, Column 12, lines 37-15) ; and
including the second candidate documents with the first candidate documents as part of determining highest-ranking documents as the top ten similar images includes the first and second best matched images (Caid275, Column 12, lines 37-39 “Optionally, a threshold can be set to retain the best matches, e.g., the top ten similar images and associated index terms 1208”; Please see the 112b above, for examination purposes this claim limitation has been construed to mean –when ranking the first candidate documents--).
Caid does not explicitly teach a stock-image query.
When read in light of the instant specification (See Paragraph [0058]), this claim limitation appears to be directed to when the query is intended to search a database of stock images. Furthermore, the identification that the images in the searched database are ‘stock’ images appears to be a recitation of non-functional descriptive material, reciting a human meaning associated with the images that does not impact the functionality of the claimed device. There is no physical distinction between a ‘stock’ image and any other image. The identification of the image being ‘stock’ is merely a human meaning that is associated with the image. Nevertheless, for the sake of compact prosecution, prior art has been applied which explicitly addresses the claim limitation.
Mousavian teaches a stock-image query (Mousavian, ¶47 “In an embodiment, the set of codes 406 allows the autoencoder to both identify an object in the image and identify a corresponding stock image. In an embodiment, the image input to the autoencoder may have different lighting, background, and occlusion features that obscure the object of interest. In an embodiment, the autoencoder identifies a synthetic image from a set of reconstructed images that illustrate various orientations of the object. In an embodiment, the encoder generates a feature embedding (or code) of the input image.”).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the image search system taught by Caid to incorporate the Machine learning techniques to detect key points identifying the image as a stock image or synthetic image as it yields the predictable results of boosting accuracy and robustness of matching (Mousavian, ¶1 “In an embodiment, machine learning techniques may be employed to detect key points or to learn better image features for matching, resulting in a significant boost to estimation accuracy and robustness”). Within the proposed combination, the detected key points (Mousavian, ¶1), including the features identifying the image as stock, or synthetic (Mousavian, ¶47), may be included as features used to generate the Context Vector (Caid275, Column 7, line 30-Column 8, line 12) which may be viewed as ‘coded information’ about the document in the context vector space (Caid709 Column 14, lines 6-13). The proposed combination would enable the system to present the coded information e.g. the identification of the image as ‘stock’ or ‘synthetic’ (Mousavian, ¶47) when the document is displayed as a search result (Caid709, Column 14, lines 11-13).
With regard to claim 9 Caid further teaches wherein the ranking as ordering the list based on similarity weight (Caid275, Column 12, lines 39-42 “The associated index terms retrieved are weighted by both similarity 1209 and frequency of occurrence 1211 and an ordered list is produced of the most relevant index terms 1213.”) is further based on relevance (Caid709; Column 18-19 “The net result is a list of the most relevant documents, independent of language”) to a [[as the search for magazine pictures, which one of ordinary skill in the art may reasonably recognize as being a ‘stock’ image (Caid275, Column 12, line 51 “Image Queries”; Column 14, lines 41-42 “Similarly, text and images can be linked, for example magazines and pictures in magazines”; Please note this claim limitation has been construed in light of Paragraph [0058] of the original specification as a query that searches for stock photos.).
Caid does not explicitly teach a stock-image query.
When read in light of the instant specification (See Paragraph [0058]), this claim limitation appears to be directed to when the query is intended to search a database of stock images. Furthermore, the identification that the images in the searched database are ‘stock’ images appears to be a recitation of non-functional descriptive material, reciting a human meaning associated with the images that does not impact the functionality of the claimed device. There is no physical distinction between a ‘stock’ image and any other image. The identification of the image being ‘stock’ is merely a human meaning that is associated with the image. Nevertheless, for the sake of compact prosecution, prior art has been applied which explicitly addresses the claim limitation.
Mousavian teaches a stock-image query (Mousavian, ¶47 “In an embodiment, the set of codes 406 allows the autoencoder to both identify an object in the image and identify a corresponding stock image. In an embodiment, the image input to the autoencoder may have different lighting, background, and occlusion features that obscure the object of interest. In an embodiment, the autoencoder identifies a synthetic image from a set of reconstructed images that illustrate various orientations of the object. In an embodiment, the encoder generates a feature embedding (or code) of the input image.”).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the image search system taught by Caid to incorporate the Machine learning techniques to detect key points identifying the image as a stock image or synthetic image as it yields the predictable results of boosting accuracy and robustness of matching (Mousavian, ¶1 “In an embodiment, machine learning techniques may be employed to detect key points or to learn better image features for matching, resulting in a significant boost to estimation accuracy and robustness”). Within the proposed combination, the detected key points (Mousavian, ¶1), including the features identifying the image as stock, or synthetic (Mousavian, ¶47), may be included as features used to generate the Context Vector (Caid275, Column 7, line 30-Column 8, line 12) which may be viewed as ‘coded information’ about the document in the context vector space (Caid709 Column 14, lines 6-13). The proposed combination would enable the system to present the coded information e.g. the identification of the image as ‘stock’ or ‘synthetic’ (Mousavian, ¶47) when the document is displayed as a search result (Caid709, Column 14, lines 11-13).
Claims 3 and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Caid in view of Goldenstein [2019/0179861].
With regard to claims 3 and 19 Caid further teaches wherein the candidate documents further include documents as retrieved images (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”; Please note this claim limitation has been construed to mean –image document--) from a [[ as image database (Caid275, Column 3, line 66 – Column 4, line 1 “Image database 106 contains images in electronic or computer-readable form.”) that include an image as the contained images (Id; Please note this claim limitation has been construed to mean –a stored image--) similar (Caid275, Column 3, lines 7-9 “Images are retrieved using any of a number of query methods (e.g., images, image portions, vocabulary atoms, index terms).”) to the query image (Caid275, Column 12, line 51 “Image Queries”).
Caid does not explicitly teach a fact-check repository.
When read in light of the instant specification (See Paragraph [0057]), this claim limitation appears to be directed to when the query is intended to search a database containing specific data (e.g. fact-check information). Furthermore, the identification that the images in the searched database are ‘fact-checked’ images appears to be a recitation of non-functional descriptive material, reciting a human meaning associated with the data being retrieved that does not impact the functionality of the claimed device. There is no physical distinction between a ‘fact-checked image and any other image. The identification of the image being ‘fact-checked is merely a human meaning that is associated with the image. Nevertheless, for the sake of compact prosecution, prior art has been applied which explicitly addresses the claim limitation.
Goldenstein teaches a fact-checked repository as a fact-checked database (Goldenstein, ¶304 “It is noted that the MetaCert Protocol already detects fake news by checking domains, websites, and news sources against the world's largest network of fact-checking databases. The classification of each news source is verified by fact-checking organizations and stored in the MetaCert Protocol registry to provide unbiased, democratically assessed information on the integrity of each website and news source”).
It would have been obvious to one of ordinary skill to which said subject matter pertains at the time the invention was filed to have implemented the search system taught by Caid to check the fact-checking databases taught by Goldenstein as it yields the predictable results of enabling the system to identify possible fake news sources (Goldenstein, ¶304). This would enable the proposed combination to identify if the image is associated with ‘fake news’ (Goldenstein, ¶4), which can aid in improving filtering techniques and slow the spread of mis-information (Goldenstein, ¶12).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to AMANDA WILLIS whose telephone number is (571)270-7691. The examiner can normally be reached Monday-Friday 8am-2pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ajay Bhatia can be reached at 571-272-3906. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/AMANDA L WILLIS/ Primary Examiner, Art Unit 2156
Prosecution Insights