DETAILED ACTION
Response to Arguments
Applicant's arguments filed with respect to claims 1-20 have been fully considered but are moot in view of the new ground(s) of rejection. The rejections are necessitated due to claim amendments.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine,
manufacture, or composition of matter, or any new and useful improvement
thereof, may obtain a patent therefor, subject to the conditions and requirements
of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to non-statutory subject matter. When reviewing independent claim 1, 10 and 16, and based upon consideration of all of the relevant factors with respect to the claim as a whole, 1-20 are held to claim an abstract idea without reciting elements that amount to significantly more than the abstract idea and is/are therefore rejected as ineligible subject matter under 35 U.S.C. 101.
The Examiner will analyze Claim 1, and similar rationale applies to independent Claims 10 and 16. The rationale, under MPEP § 2106, for this finding is explained below:
The claimed invention (1) must be directed to one of the four statutory categories, and (2) must not be wholly directed to subject matter encompassing a judicially recognized exception, as defined below. The following two step analysis is used to evaluate these criteria.
Step 1: Is the claim directed to one of the four patent-eligible subject matter categories: process, machine, manufacture, or composition of matter?
When examining the claim under 35 U.S.C. 101, the Examiner interprets that the claims is related to a process since the claim is directed to a method.
Step 2a, Prong 1: Does the claim wholly embrace a judicially recognized exception, which includes laws of nature, physical phenomena, and abstract ideas, or is it a particular practical application of a judicial exception?
The Examiner interprets that the judicial exception applies since Claim 1 limitation of receiving a relevance score from the cross-modal matching service for at least one image in the collection of images; and selecting the at least one image to associate with the input text based on the relevance in the context in which the input text and the at least one image is to be presented are directed to an abstract.
The limitations creating relevance score and selecting an image could be performed by a person (mental process).
Accordingly, the "mental processes" abstract idea grouping is defined as concepts performed in the human mind, and examples of mental processes include observations, evaluations, judgments, and opinions, The courts consider a mental process (thinking) that "can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 (2012) ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same).
If/when the claim recites a judicial exception (i.e., an abstract idea enumerated in MPEP § 2106.04(a), a law of nature, or a natural phenomenon), the claim requires further analysis in Prong Two.
Step 2a, Prong 2: Does the claim recite additional elements that integrate the judicial exception into a practical application?
The additional claim limitations providing input text and a collection of images into a cross-modal matching service is nothing more than insignificant extra solution activity.
A cross-modal is used to generally apply the abstract idea without limiting how it functions.
Step 2b: If a judicial exception into a practical application is not recited in the claim, the Examiner must interpret if the claim recites additional elements that amount to significantly more than the judicial exception.
The Examiner interprets that the Claims do not amount to significantly more since the Claims are generally linking the use of the judicial exception to a particular technological environment or field of use, e.g., a claim describing how the abstract idea of hedging could be used in the commodities and energy markets, as discussed in Bilski v. Kappos, 561 U.S. 593, 595, 95 USPQ2d 1001, 1010(2010) or a claim limiting the use of a mathematical formula to the petrochemical and oil-refining fields, as discussed in Parker v. Flook, 437 U.S. 584, 588-90, 198 USPQ 193, 197-98 (1978) (MPEP § 2106.05(h)).
Furthermore, the generic computer components of the processor/memory recited as performing generic computer functions that are well-understood, routine and conventional activities amount to no more than implementing the abstract idea with a computerized system.
Claims 2-9, 11-15, and 17-20 depending on the independent claims include all the limitation of the independent claim. The Examiner finds that Claims 2-8, 22-26, and 28-32 does not state significantly more since the claim only recites additional steps for analyzing video using machine learning model.
Thus, claims 1-20 recite the same abstract idea and therefore are not drawn to the eligible subject matter as they are directed to the abstract idea without significantly more.
Therefore, all claims are rejected under 35 U.S.C. 101.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1, 10 and 16 are rejected under 35 U.S.C. 102 (a)(2) as being anticipated by Maheshwari et al. (Pub. No. US 2021/0342389).
Regarding claim 1, Maheshwari teaches providing input text (query) and a collection of images (set of relevant images) into a cross-modal matching service ‘Para. 43 “the architecture 600 receives a (query, image) pair as input and processes the pair to output a score that indicates the relevance of the image in the context of the query relative to color intent”; Para. 59 “the identified set of relevant images is processed according to the ranker process described with respect to FIG. 6 to determine a relevance measure (e.g., relevance score)”];
Receiving/determining a relevance score from the cross-modal matching service (ranker process) for at least one image in the collection of images (a set of relevant images), [Para. 59 “the identified set of relevant images is processed according to the ranker process described with respect to FIG. 6 to determine a relevance measure (e.g., relevance score).”] wherein the relevance score indicates a relevance of the at least one image with respect to the input text in a context in which the input text and the at least one image is to be presented (positioned) [Para. 43 “The architecture 600 receives a (query, image) pair as input and processes the pair to output a score that indicates the relevance of the image in the context of the query relative to color intent.”; Para. 60 “more relevant images can be positioned at the top of a list of relevant images while less relevant images are positioned at the bottom of the list”]; and Selecting (ordered) the at least one image to associate (returned) with the input text based on the relevance in the context (list of relevant images) in which the input text and the at least one image is to be presented (positioned) [Para. 60].
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 10, and 16 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089).
Regarding claim 1, Zhu teaches providing input text and a collection of images (candidate images received and stored) into a cross-modal/algorithm matching service [Abstract; para. 42 and 20; fig. 6 step 602, 604 and corresponding description]; receiving a relevance/correlation score from the cross-modal matching service for at least one image in the collection of images, wherein the relevance score indicates a relevance of the at least one image with respect to the input text [Para. 42, fig. 6 step 608 and corresponding description]; and selecting the at least one image to associate with the input text [Para. 42; fig. 6 step 612, and related description].
However, Zhu doesn’t explicitly teach the rest of claim limitations.
HU teaches wherein the relevance/similarity score indicates a relevance of the at least one image with respect to the input text in a context (matching result) in which the input text/query and the at least one image is to be presented/present [Para. 60 “the similarity function module 130 may compute a similarity score between an input query and each of a plurality of input images, and may provide the similarity scores to the image selection module 140”; “Para. 61 “The candidate images are passed into the fine search model 200, and the fine search model 200 may select at least one image from the candidate images and present the selected at least one image as a matching result of the input query”. since the term “context” is not explicitly defined, the “matching result” reads on it.] and selecting the at least one image (final image) to associate/corresponding with the input text/query based on the relevance in the context in which the input text and the at least one image is to be presented/displayed [Para. 79 “at least one final image corresponding to the search query is selected from the candidate images, based on the similarity scores between search query and each of the candidate images, via the fine search model”; Para. 80 “at least one final image is displayed on a user device”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu to teach the claim limitations, feature as taught by LU; because the modification provides multimodal content retrieval system that reduces the retrieval latency with minimal loss in ranking accuracy for on-device language-based image retrieval
Claims 10 and 16 are rejected for the same reason as claim 1 above. Furthermore, Zhu teaches a system, a processor, and a memory storing instructions [fig. 7 and related description].
Claims 2, 11, and 17 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089) further in view of Li et al. (Pub. No. US 20200250537).
Regarding claims 2, 11, and 17, Zhu in view of Hu doesn’t explicitly teach the claim limitation.
However, Li teaches wherein the cross-modal (image/text embedding model) matching service includes a trained machine learning model, wherein the trained machine learning model was trained on a first dataset including pairings of images and text (query image training example), and the dataset includes a positive example (positive label) of relevance of a first training dataset image to first paired text when a desired interaction (selected) with respect to the first training dataset image and the first paired text has been observed in the context (search result) in which they were presented [Para. 51, 52, and 70], whereby relevance between the at least one image and the input text is based on expectations of achieving the desired interaction in the context (search result) in which they are presented. [Para. 86 and 52].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu in view of Hu to train the model with text, image and positive example, feature as taught by Li; because the modification enables the system to improve determination between an image and input text by training the model using observed user interaction signals as the ground truth of relevance.
Claims 3 and 12 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089) further in view of Li et al. (Pub. No. US 20200250537) further in view of Kulkarni et al. (Pub. No. US 2018/0004847).
Regarding claims 3 and 12, Zhu in view of HU further in view Li doesn’t explicitly teach the claim limitation.
However, Kulkarni teaches wherein the first training dataset image and the first paired text occur with respect to an item of invitational content (sponsored content) that was served to a user device by a content delivery system, and the desired interaction is a click or conversion of the item of invitational content on the user device [Para. 4, 27, and 28].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu in view of HU further in view Li to train the model with negative example, feature as taught by Kulkarni; because the modification enables the system to dynamically optimize content for target user.
Claims 4, 13, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089) further in view of Li et al. (Pub. No. US 20200250537) further in view of KLOTZ, JR. (Pub. No. US 2017/0329857).
Regarding claims 4, 13 and 18 Zhu in view of HU further in view Li doesn’t explicitly teach the claim limitation.
However, KLOTZ teaches wherein the first training dataset image was presented in an item of invitational content and the first paired text was received in a search input of an App store, wherein the App store is the context in which they were presented [Para. 93, 94, 98, 105; fig. 4, 5 and related description; Para. 23 “In some examples, native applications may be installed on a user device 200 prior to the purchase of the user device 200. In other examples, a user device 200 may download a native application from a digital distribution platform such as the APP STORE® digital distribution platform developed by Apple Inc. or the GOOGLE PLAY® digital distribution platform developed by Google Inc. In these examples, the user device 200 downloads and installs the application at the request of a user”; Para. 7 “the application features may also include digital media related to the application 110, such as images (e.g., icons associated with the application and/or screenshots of the application) or videos (e.g., a sample video of the application)”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu in view of HU further in view Li to teach the claim limitation, feature as taught by KLOTZ; because the modification enables the system to personalize search results based on a device profile of user device.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089) further in view of Li et al. (Pub. No. US 20200250537) further in view of Baker et al. (Patent No. US 8,503,769).
Regarding claim 5, Zhu in view of HU further in view Li doesn’t explicitly teach the claim limitation.
However, Baker teaches wherein the first dataset includes a negative example of relevance of the first training dataset image to paired text, wherein the negative example was derived from remixing a first image with text that was not associated with the first image [Col. 3 lines 55-60; Col. 5 lines 44-65].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu in view of HU further in view Li to train the model with negative example, feature as taught by Baker; because the modification enables the system to determine information about the images from the web page.
Claim 6 is rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089) further in view of Li et al. (Pub. No. US 20200250537) further in view of Wright et al. (Pub. No. US 2007/0156887).
Regarding claim 6, Zhu in view of HU further in view Li doesn’t explicitly teach the claim limitation.
Wright teaches wherein the trained machine learning model was trained on a second dataset (rating data) including second pairings of images and text, and the second dataset includes human labeled (rating) positive examples of relevance of a second training dataset image to second paired text, and human labeled negative examples of relevance of a third training dataset image to third paired text [Para. 27, 31, 47 and 48].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu in view of HU further in view Li to train the model with negative example, feature as taught by Wright; because the modification enables the system to determine information about the images from the web page.
Claims 7, 14, and 19 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089) further in view of Kaftan et al. (Pub. No. US 2015/0324868).
Regarding claims 7, 14, and 19 Zhu in view of HU doesn’t explicitly teach the claim limitation.
However, Kaftan teaches wherein the selecting the at least one image to associate with the input text further comprises: receiving, the input text as a search query entered into an App store: generating invitational content (an ad) including the at least one image and the input text, wherein the at least one image is a most relevant image to the input text in the collection of images, wherein the input text is a search term expected to be used in an App store, and the collection of images are images relevant to an App to be presented by the invitational content, presenting the invitational content in response to the search query [Abstract, Para. 3, 4, 7, 26 and 51 “The application features may also include digital media related to the application, such as images (e.g., icons associated with the application and/or screenshots of the application)”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu in view of HU to teach the claimed limitation, feature as taught by Kaftan; because the modification enables the system to determine a query categorization of a search query.
Claim 8 is rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089) further in view of Kaftan et al. (Pub. No. US 2015/0324868) and further in view of Kulkarni et al. (Pub. No. US 2018/0004847).
Regarding claim 8, Zhu in view of HU and Kulkarni doesn’t explicitly teach the claim limitation.
However, Kulkarni teaches wherein the at least one image is the image from the collection of images that is most likely to result in a desired interaction (click) when the at least one image is included in the invitational content when it is served to a user terminal in response to the App store receiving the input text as the search term [Para. 13, claim 7, Para. 16 “The online system 130 selects the creative with the highest prediction score from each category of creatives and combines the selected creatives of each type to create the DCO content item for the target user”].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu in view of HU and Kaftan to teach the claim limitation, feature as taught by Kulkarni; because the modification enables the system to dynamically optimize content for target user.
Claims 9, 15, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Pub. No. US 2013/0110620) in view of HU et al. (Pub. No. US 2023/0237089) further in view of LIU et al. (Pub. No. US 2012/0117051).
Regarding claims 9, 15, and 20, Zhu in view of HU doesn’t explicitly teach the claim limitation.
However, LIU teaches wherein the providing input text and the collection of images into the cross-modal matching service includes receiving the input text from search keywords received into a search input [Para. 2-4]; wherein the collection of images is included in respective items of invitational content [Para. 60, 61, fig. 4 and related description]; and wherein the selecting the at least one image to associate with the input text includes selecting at least one of the respective items of invitational content to be displayed in an App store along with search results that are relevant to the input text [para. 26, 75-76].
It would have been obvious to one of ordinary skill in the art before the effective filing date to modify Zhu in view HU to teach the claimed limitation, feature as taught by LIU; because the modification enables the system to determine result based on multiple modes queries.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to SOLOMON G BEZUAYEHU whose telephone number is (571)270-7452. The examiner can normally be reached on Monday-Friday 10 AM-8 PM.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of an application may be obtained from the Patent Application Information Retrieval (PAIR) system. Status information for published applications may be obtained from either Private PAIR or Public PAIR. Status information for unpublished applications is available through Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 888-786-0101 (IN USA OR CANADA) or 571-272-4000.
/SOLOMON G BEZUAYEHU/
Primary Examiner, Art Unit 2666