DETAILED ACTION
Claim Interpretation
Claims 15-20 recite “one or more hardware storage devices storing instructions…”. A storage device cannot a signal and thus claims 15-20 are eligible under 35 U.S.C. § 101. This is further supported by the specifications at ¶128 “Computer storage media (aka “hardware storage device”) are computer-readable hardware storage devices, such as RAM, ROM, EEPROM, CD-ROM, solid state drives (“SSD”) that are based on RAM, Flash memory, phase-change memory (“PCM”), or other types of memory, or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code means in hardware in the form of computer-executable instructions, data, or data structures and that can be accessed by a general-purpose or special-purpose computer”.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The following is a quotation of pre-AIA 35 U.S.C. 112, fourth paragraph:
Subject to the following paragraph [i.e., the fifth paragraph of pre-AIA 35 U.S.C. 112], a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
Claims 16-20 are rejected under 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph, as being of improper dependent form for failing to further limit the subject matter of the claim upon which it depends, or for failing to include all the limitations of the claim upon which it depends. Independent claim 15 is directed to “one or more hardware storage devices storing instructions executable by one or more processors of a system”. However, claims 16-20 depend from claim 15 by reciting “the system of claim 15”. Because claim 15 is directed to hardware storage devices rather than a system, claims 16-20 do not further limit the subject matter of claim 15 as required by 35 U.S.C. 112(d) or pre-AIA 35 U.S.C. 112, 4th paragraph.
Applicant may cancel the claim(s), amend the claim(s) to place the claim(s) in proper dependent form, rewrite the claim(s) in independent form, or present a sufficient showing that the dependent claim(s) complies with the statutory requirements.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-2, 5-7, 8-9, 12-14, 15-16, 19-20 is/are rejected under 35 U.S.C. 102(a)(2) as being anticipated by Faieta et al (US 20230137774).
Regarding claim 1, Faieta discloses a system for facilitating image search using object state distributions (abstract: Systems and methods for image retrieval; identify an image in a database based on metadata for the image including an object label corresponding to the extracted entity and an object color embedding in the color embedding space corresponding to the object label; and provide image information for the image to the user based on the metadata), the system comprising:
one or more processors (Fig. 4 processor unit 405); and
one or more hardware storage devices storing instructions that are executable by the one or more processors (¶70-72) to configure the system to:
receive a first search query element, the first search query element indicating or representing one or more objects (¶74 a search query is input to color name entity recognition network 500; For example, the search query is “light brown cow on green background”; e.g. cow is the object);
receive a second search query element, the second search query element indicating one or more object states (¶74 For example, the search query is “light brown cow on green background”; e.g. cow is the object; e.g. light brown and green background as object states; color name entity recognition network 500 detects that “light brown” and “green” are color phrases in the search query); and
perform an image search operation based on the first search query element and the second search query element (¶78 Search component 510 is configured to match the entity in the search query to the object in the image, and to determine that the entity color embedding is similar to the object color embedding), the image search operation comprising:
obtaining an object state distribution based upon the first search query element and the second search query element (¶76 Multi-modal color encoder 505 is configured to generate an entity color embedding for the color phrase describing an entity in a search query, and to generate an object color embedding for the object based on an object mask; ¶78 Search component 510 identifies an image in a database 515 based on metadata for the image including an object label corresponding to the extracted entity and an object color embedding in the color embedding space corresponding to the object label);
determining a plurality of distance measures, the plurality of distance measures comprising a respective distance measure between the object state distribution and each of a plurality of object state distributions (¶64 In some examples, search component 440 calculates a distance between the entity color embedding and the object color embedding in the color embedding space. For example, the distance could be a Euclidean distance in a vector space. Search component 440 compares the distance to a threshold value, where the image is identified based on the comparison. According to some embodiments, search component 440 determines that the entity color embedding is within a threshold distance of the object color embedding in a color embedding space), each of the plurality of object state distributions being associated with a respective image set of an image index (¶39 database 120 includes a set of images indexed for image search, wherein each of the plurality of images comprises image metadata including labels and color embedding information corresponding to the labels); and
ranking each of the respective image sets of the image index based on the respective distance measures of the plurality of distance measures (¶102 The image retrieval apparatus includes image understanding, query understanding, multi-modal bridge, and search matching and ranking; ¶128 The candidate images that match in color are ranked higher than candidate images that do not match in color. Similarly, a candidate image whose color is different from the color phrase in the search query can be removed from the search results; ¶132 the system identifies an image in a database based on metadata for the image including an object label corresponding to the extracted entity and an object color embedding in the color embedding space corresponding to the object label).
Regarding claim 2, Faieta discloses the system of claim 1, wherein the first search query element comprises text input indicating the one or more objects (¶2 a user may provide a text description of an image as a query; ¶34 During query search (i.e., online stage), the query color phrase is mapped to a corresponding entity color embedding using text-to-color embedding method).
Regarding claim 5, Faieta discloses the system of claim 1, wherein the second search query element comprises text input indicating the one or more object states (¶2 a user may provide a text description of an image as a query; ¶74 For example, the search query is “light brown cow on green background”; e.g. cow is the object; e.g. light brown and green background as object states; color name entity recognition network 500 detects that “light brown” and “green” are color phrases in the search query).
Regarding claim 6, Faieta discloses the system of claim 1, wherein the object state distribution comprises a plurality of object state vectors (¶43 At operation 210, the system embeds the search query in an embedding space. In some cases, more than one vector representation can be generated for a search query; ¶64 In some examples, search component 440 calculates a distance between the entity color embedding and the object color embedding in the color embedding space. For example, the distance could be a Euclidean distance in a vector space.).
Regarding claim 7, Faieta discloses the system of claim 6, wherein the plurality of object state vectors comprises a plurality of embeddings generated by one or more object state description modules (¶43 At operation 210, the system embeds the search query in an embedding space. In some cases, more than one vector representation can be generated for a search query; ¶64 In some examples, search component 440 calculates a distance between the entity color embedding and the object color embedding in the color embedding space. For example, the distance could be a Euclidean distance in a vector space.).
Regarding claim(s) 8-9 and 12-14 (drawn to a method):
The rejection/proposed combination of Faieta, explained in the rejection of system claim(s) 1-2 and 5-7, anticipates/renders obvious the steps of the method of claim(s) 8-9 and 12-14 because these steps occur in the operation of the proposed combination as discussed above. Thus, the arguments similar to that presented above for claim(s) 1-2 and 5-7 is/are equally applicable to claim(s) 8-9 and 12-14.
Regarding claim(s) 15-16 and 19-20 (drawn to a CRM):
The rejection/proposed combination of Faieta, explained in the rejection of system claim(s) 1-2 and 5-6, anticipates/renders obvious the steps of the computer readable medium of claim(s) 15-16 and 19-20 because these steps occur in the operation of the proposed combination as discussed above. Thus, the arguments similar to that presented above for claim(s) 1-2 and 5-6 is/are equally applicable to claim(s) 15-16 and 19-20. See Faieta ¶70-72.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 3-4, 10-11 and 17-18 is/are rejected under 35 U.S.C. 103 as being unpatentable over Faieta as applied to claim 1, 8 and 15 above, and further in view of Huang et al (US 20190354609).
Regarding claim 3, Faieta discloses the system of claim 1, but fails to teach where Huang teaches wherein the first search query element comprises image input representing the one or more objects (¶26 the visual search system 130 may receive a query image from a user of a client 110,112,114 and search a corpus of product database images 190 (or any corpus of searchable images) to find matching images for the query image.).
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of wherein the first search query element comprises image input representing the one or more objects from Huang into the system as disclosed by Faieta. The motivation for doing this is to improve visual search systems for generating image search results of visually similar products and images.
Regarding claim 4, Faieta discloses the system of claim 1, but fails to teach where Huang teaches wherein the first search query element comprises a user selection of one or more objects represented in a set of one or more images (¶29 a user can specify a crop box 204, as shown in this example, to select a product of interest and to obtain visually similar products or images).
Therefore, it would have been obvious to one with ordinary skill in the art before the effective filing date of the invention to have implemented the teaching of wherein the first search query element comprises a user selection of one or more objects represented in a set of one or more images from Huang into the system as disclosed by Faieta. The motivation for doing this is to improve visual search systems for generating image search results of visually similar products and images.
Regarding claim(s) 10-11 (drawn to a method):
The rejection/proposed combination of Faieta and Haung, explained in the rejection of system claim(s) 3-4, anticipates/renders obvious the steps of the method of claim(s) 10-11 because these steps occur in the operation of the proposed combination as discussed above. Thus, the arguments similar to that presented above for claim(s) 3-4 is/are equally applicable to claim(s) 10-11.
Regarding claim(s) 17-18 (drawn to a CRM):
The rejection/proposed combination of Faieta and Haung, explained in the rejection of system claim(s) 3-4, anticipates/renders obvious the steps of the computer readable medium of claim(s) 17-18 because these steps occur in the operation of the proposed combination as discussed above. Thus, the arguments similar to that presented above for claim(s) 3-4 is/are equally applicable to claim(s) 17-18. See Faieta ¶70-72.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to KEVIN KY whose telephone number is (571)272-7648. The examiner can normally be reached Monday-Friday 9-5PM.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Vincent Rudolph can be reached at 571-272-8243. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/KEVIN KY/ Primary Examiner, Art Unit 2671
Prosecution Insights