DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA . In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
Information Disclosure Statement (IDS)
The information disclosure statement filed July 9, 2025, fails to comply with 37 CFR 1.98(a)(3)(i) because it does not include a concise explanation of the relevance, as it is presently understood by the individual designated in 37 CFR 1.56(c) most knowledgeable about the content of the information, of each reference listed that is not in the English language. It has been placed in the application file, but the information referred to therein has not been considered.
Specification
The specification has not been checked to the extent necessary to determine the presence of all possible minor errors. Applicant’s cooperation is requested in correcting any errors of which applicant may become aware in the specification.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more.
When considering subject matter eligibility under 35 U.S.C. 101, it must be determined whether the claim is directed to one of the four statutory categories of invention (i.e., process, machine, manufacture, or composition of matter) (step 1). If the claim does fall within one of the statutory categories, it must then be determined whether the claim is directed to a judicial exception (i.e., law of nature, natural phenomenon, and abstract idea) (step 2A), and if so, it must additionally be determined whether the claim is a patent-eligible application of the exception (step 2B). Alice Corp. Pty. Ltd. v. CLS Bank Int'l, 134 S. Ct. 2347, 189 L. Ed. 2d 296, 2014 U.S. LEXIS 4303, 110 U.S.P.Q.2D (BNA) 1976, 82 U.S.L.W. 4508, 24 Fla. L. Weekly Fed. S 870, 2014 WL 2765283 (U.S. 2014); MPEP 2106.
Step 1:
In the instant case claims 1-13 are directed to a machine and claims 14-20 are directed to a process. All claims are therefore within statutory categories. See MPEP 2106.03, Eligibility Step 1.
Step 2A, Prong 1:
These claims also recite, inter alia,
“providing, by a communication interface via a network, a search interface of a user interface to a plurality of user…s; storing, … a product database and a set of computer executable instructions, wherein the product database comprises a plurality of product records each associated with a set of search vectors; executing the set of computer executable instructions; providing the search interface including a plurality of search modality options, wherein the plurality of search modality options comprises at least two of a text search, an image search, a video search, an audio search, a barcode search, a color search, or a URL search; receiving a first query in a first modality of the plurality of search modality options from the search interface; generating, via a language model, a product results list based on the first query and the product database; receiving a second query in a second modality of the plurality of search modality options from the search interface; and updating, via the language model, the product results list based on the first query and the second query.” Claim 14.
With recited additional elements reserved for consideration alone and all together combined with their recited role(s) in the claim under step 2A prong two, a careful analysis of the remaining limitations above results in the conclusion that each on its own recites an abstract idea and in combination they simply recite a more detailed abstract idea. The recited abstract idea falls within the grouping of abstract ideas described as certain methods of organizing human activity, for example commercial interactions (including advertising, marketing or sales activities or behaviors). See MPEP 2106.04(a); Eligibility Step 2A1. The claims must therefore be analyzed under the second prong of Eligibility Step 2 (Step 2A2; MPEP 2106.04(d)).
Step 2A, Prong 2:
In order to address prong 2 (MPEP 2106.04(d), Eligibility Step2A2) we must identify whether there are any additional elements beyond the abstract ideas and determine whether those additional elements (if there are any) integrate the abstract idea into a practical application. MPEP 2106.04(d), Eligibility Step 2A2. The additional elements in claims 1-13 are a plurality of user devices and a control circuit configured to execute computer executable instructions. The additional elements in claims 14-20 are a plurality of user devices. These additional elements have been considered individually, in combination, and altogether as a whole together with the functions they perform, e.g., the plurality of user devices perform no particularly claimed active function, merely serving as recipient of communicated data in place of the user in the process, while the control circuit of claims 1-13 by way of its execution of computer executable instructions, is broadly and generally recited as performing all steps in terms of the intended results of functionally nonspecific activities. This additional element does not integrate the judicial exception into a practical application because the claim recitations amount to no more than mere instructions to apply the exception using a generic computer component. The claim is otherwise almost entirely a recitation of abstract ideas. The substantive process is recited only by descriptions of abstract intended results of the steps without indicating any particular functional acts performed by any device or structural element to perform the steps or otherwise obtain the intended results. The additional elements do not improve the functioning of any computer or other technology or technical field, they do not apply the judicial exception with or by use of a particular machine, they do not transform or reduce a particular article to a different state or thing, and they fail to apply or use the judicial exception beyond generally linking the use of the judicial exception to a particular technological environment. See MPEP 2106.05.
If the disclosure describes any improvements to the functioning of a computer or to any other technology or technical field this improvement would need to be identifiable as the subject matter appearing in the claims. An indication that the claimed invention provides an improvement can include a discussion in the specification that identifies technical improvements realized by the claim over the prior art. The disclosure must provide sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement. MPEP 2106.05(a).
Claim limitations can integrate a judicial exception into a practical application by implementing the judicial exception with or using it in conjunction with a particular machine or manufacture that is integral to the claim. A general purpose computer that applies a judicial exception by use of generic computer functions does not qualify as a particular machine. Ultramercial, Inc. v. Hulu, LLC, (Fed. Cir. 2014); MPEP 2106.05(b),(f). There are no particular machines or manufactures identified in the present claims. Claimed elements that are not abstract are identified broadly and generally as applying the method, and the method itself is described only by way of the intended functional results of unidentified activities, without reference to any particular operations or specific functions performed by any particularly identified machines, and without reference to its use in conjunction with any particular item of manufacture.
The claims do not affect the transformation or reduction of a particular article to a different state or thing. Changing to a different state or thing means more than simply using an article or changing the location of an article. A new or different function or use can be evidence that an article has been transformed. Purely mental processes in which data, thoughts, impressions, or human based actions are "changed" are not considered a transformation. MPEP 2106.05(c).
The claims do not apply or use the judicial exception in any other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment. As a result the claim as a whole appears to be a drafting effort designed to monopolize the exception. MPEP 2106.05(e),(h).
The additional elements have not been found to integrate the abstract idea into a practical application.
Step 2B:
Although the additional elements have not been found to integrate the abstract idea into a practical application the claims could still be eligible if they recite additional elements that amount to an inventive concept (“significantly more” than the judicial exception). MPEP 2106.05, Eligibility Step 2B.
The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the sparse additional elements of the claim are mere props supporting instructions to implement an abstract idea or other exception on a computer. MPEP 2106.05(f). The claims invoke computers or other machinery merely as tools to perform an abstract process. Simply adding a general purpose computer or computer components after the fact to an abstract idea does not provide significantly more. MPEP 2106.05(f)(2); see also OIP Techs., Inc. v. Amazon.com, Inc., 788 F.3d 1359, 2015 U.S. App. LEXIS 9721, 115 U.S.P.Q.2D (BNA) 1090 (Fed. Cir. 2015) (“relying on a computer to perform routine tasks more quickly or more accurately is insufficient to render a claim patent eligible.”). The claims fail to provide a technical solution to a technical problem created by the use of the surrounding technology. Limitations that amount to merely indicating a field of use or technological environment in which to apply a judicial exception do not amount to significantly more than the exception itself. See Ret. Capital Access Mgmt. Co. v. U.S. Bancorp, 611 Fed. Appx. 1007, 2015 U.S. App. LEXIS 14351 (Fed. Cir. 2015) (“It may be very clever; it may be very useful in a commercial context, but they are still abstract ideas,” said Circuit Judge Alan Lourie.). MPEP 2106.05(h).
Finally, it is reiterated that the remaining dependent claims 2-13 and 15-20 do not contribute any additional elements other than those already discussed and do not add "significantly more" to establish eligibility because they merely recite additional abstract ideas that further identify the data and the manipulation of the data used in implementing the abstract idea. A more detailed abstract idea is still abstract. PricePlay.com, Inc. v. AOL Adver., Inc., 627 Fed. Appx. 925, 2016 U.S. App. LEXIS 611, 2016 WL 80002 (Fed. Cir. Jan. 7, 2016) (in addressing a bundle of abstract ideas stacked together during oral argument, U.S. Circuit Judge Kimberly Moore said, "All of these ideas are abstract…. It’s like you want a patent because you combined two abstract ideas and say two is better than one.").
All of the above leads to the conclusion that additional claim elements do not provide meaningful limitations to transform the claimed subject matter into significantly more than an abstract idea. MPEP 2106.05; Eligibility Step 2B. As a result the claims are rejected under 35 USC 101 as being directed to non-statutory subject matter because they recite an abstract idea without being directed to a practical application, and they do not amount to significantly more than the abstract idea. MPEP 2106.05, supra..
The preceding analysis applies to all statutory categories of invention. Accordingly, claims 1-20 are rejected as ineligible for patenting under 35 USC 101 based upon the same analysis.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1-3 and 5-20 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Zhu et al. (Patent No.: US 12,210,516 B1).
Zhu teaches a) a system for providing product search results, b) providing a search interface to a plurality of user devices, c) a multimodal search interface, and d) generating product records as search results via a language model, and discloses regarding
Claim 1. A system for providing product search results, the system comprising: ● a communication interface configured to provide, via a network, a search interface of a user interface to a plurality of user devices (see at least Zhu figs. 6A-F, c1:5-18 “Users are increasingly utilizing electronic devices to research, locate, and obtain various types of information. For example, users may utilize a search engine to locate information about various items”); ● a computer readable storage memory storing a product database and a set of computer executable instructions, wherein the product database comprises a plurality of product records each associated with a set of search vectors (see at least Zhu figs. 4,8, c5:60-67 “source catalog 202 may be processed to populate the MIM index 112 with different combined feature vectors for given items within the source catalog 202”); and ● a control circuit configured to execute the set of computer executable instructions (see at least Zhu fig.11, c2:30-67 “FIG. 1 illustrates components of an example computing environment 100 that can be used to implement aspects of the various embodiments. … client device can be any appropriate computing device capable of generating such inputs, as may include a smartphone, desktop, set-top box 45 (e.g., Fire TV), voice-enabled device (e.g., Echo), or tablet computer,” c7:25-35 “interaction environment in some embodiments includes a collection of network-accessible services executed on computer hardware”), which causes the control circuit to: ● provide the search interface including a plurality of search modality options, wherein the plurality of search modality options comprises at least two of a text search, an image search, a video search, an audio search, a barcode search, a color search, or a URL search (see at least Zhu c2:9-29 “the user may present a first search query using a first modality, such as an image…. additional modalities, such as adding textual or auditory information to refine the initial search query. These different search modalities may then be processed by a training machined learning system, which may align different representations (e.g., text and image, text and audio, image and image, etc.),” c2:30-67 “"first" and "second" are used to label the inputs and not to identify the order in which they are used or processed by the system, as the first input may be provided first or the second input may be provided first…. inputs may include a textual input (e.g., a search string typed in by a user), an audio input (which may be converted to a textual input using one or more natural language processing systems or may be processed as an auditory input), an image input, a video input, or a combination thereof, such as text-based input with an accompanying audio or image sample”); ● receive a first query in a first modality of the plurality of search modality options from the search interface (see at least Zhu c2:9-29 “the user may present a first search query using a first modality, such as an image”); ● generate, via a language model, a product results list based on the first query and the product database (see at least Zhu figs.7, 9, c2:9-29 “the user may present a first search query using a first modality, …. The present disclosure may enable refinements that use one or more additional modalities, such as adding textual or auditory information to refine the initial search query. These different search modalities may then be processed by a training machined learning system, … the trained system may establish a correspondence between matching visual and language concepts, thereby providing an improved result to the search query,” c6:64-c7:10 “training processes that include … image-text contrastive learning (ITC) on the image and text encoders 302A, 302B, 304, image-text matching (ITM), and masked language modeling (MLM) on the multimodal encoders 306A”); ● receive a second query in a second modality of the plurality of search modality options from the search interface (see at least Zhu figs.7, 9, c2:9-29 “refinements that use one or more additional modalities, such as adding textual or auditory information to refine the initial search query”); and ● update, via the language model, the product results list based on the first query, the second query, and the product database (see at least Zhu figs.1, 5, 7, 9, c7:60-67 “one or more language models may be utilized to weight the search server/engine 410 to influence the results returned from the query”).Claim 2. The system of claim 1, wherein generating the product results list comprises: ● receiving embeddings associated with the first query from the language model (see at least Zhu abstract “map features of two or more search queries to a common embedding space. … The combined feature vector aligns different search modalities within the common embedding space that may be executed against an index,” c2:20-30 “different search modalities may then be processed by a training machined learning system, which may align different representations (e.g., text and image, text and audio, image and image, etc.) within a common embedding space,” c3:20-39 “first input 102 may be computed, for example using a deep embedding model,” c3:40-65 “Both the first input 102 and the second input 104 are provided to a multimodal environment 106, which may include one or more trained machine learning systems that aligns both image and text representations (among other representations) within a common embedding space”); and ● identifying a plurality of products from the plurality of product records by matching the embeddings associated with the first query with search vectors associated with products in the product database (see at least Zhu figs. 4-5, 8-9, c2:1-30 “search inputs may be processed through a trained machine learning system to generate independent feature vectors. These feature vectors are then combined to generate a combined feature vector (e.g., shared feature vector), which may be mapped within a shared representation space. This combined feature vector may then be processed against an index to generate a set of results responsive to the initial search query …. different search modalities may then be processed by a training machined learning system, which may align different representations (e.g., text and image, text and audio, image and image, etc.) within a common embedding space. As a result, the trained system may establish a correspondence between matching visual and language concepts, thereby providing an improved result to the search query”).Claim 3. The system of claim 1, wherein the first query is converted to a text string or an encoded image as input to the language model (see at least Zhu fig.3, c2:45-55 “inputs may include a textual input (e.g., a search string typed in by a user), an audio input (which may be converted to a textual input using one or more natural language processing systems or may be processed as an auditory input), an image input, a video input, or a combination thereof, such as text-based input with an accompanying audio or image sample,” c4:25-40 “the initial input query may be of a first modality and converted to a different modality. By way of example, an auditory input may be evaluated using one or more natural language processors and then converted to a textual input. As another example, an image input that includes text may be split or otherwise segmented into an image input and a textual input, where the text is extracted from the image. As yet another example, a video input may be segmented into one or more frames and used as one or more image inputs and/or the audio track of the video input is received and converted into a textual input”. Please note: the claim language consisting of alternative limitations separated by “or” does not result in further limitation beyond a single alternative because beyond the presence of a single alternative it merely represents a contingency that is not required. Applicant is reminded that optional or conditional elements do not narrow the claims because they can always be omitted. See e.g. MPEP §2111.04 "Claim scope is not limited by claim language that suggests or makes optional but does not require steps to be performed, or by claim language that does not limit a claim to a particular structure."; and In re Johnston, 435 F.3d 1381,77 USPQ2d 1788, 1790 (Fed. Cir. 2006) ("As a matter of linguistic precision, optional elements do not narrow the claim because they can always be omitted.")).Claim 5. The system of claim 1, wherein the product results list is updated based on: ● weighting embeddings associated with the first modality with a first weighted vector (see at least Zhu abstract “Independent feature vectors may be generated for each of the initial input query and the refinement query, may be weighted, and then may be combined to form a combined feature vector. The combined feature vector aligns different search modalities within the common embedding space that may be executed against an index”); ● weighting embeddings associated with the second modality with a second weighted vector (see at least Zhu abstract “Independent feature vectors may be generated for each of the initial input query and the refinement query, may be weighted, and then may be combined to form a combined feature vector. The combined feature vector aligns different search modalities within the common embedding space that may be executed against an index”); and ● identifying products in the product database based on the weighted embeddings associated with the first modality and the second modality (see at least Zhu figs. 7-9, c2:1-10 “combined feature vector may then be processed against an index to generate a set of results responsive to the initial search query and refinements”).Claim 6. The system of claim 1, wherein the control circuit further: ● receives a third query from the search interface (see at least Zhu figs.7, 9, c1:10-15 “If the user is unsatisfied, then additional refinements may be presented,” c2:9-29 “refinements that use one or more additional modalities”. Please note: examiner's position is that the prior art describes this limitation and that multiple refinements and repetition of the step of further refining the query is inherent in the prior art. Examiner further posits that this recitation is merely the repetition of a step to achieve the same result as previously achieved. Such repetition has been held to involve only routine skill in the art. A person of ordinary skill in the art would have considered it obvious to repeat the same step to achieve the same desired result. In fact even a lay-person with no skill in the art would expect the same result to follow from a repetition of the same step.); and ● further updates, via the language model, the product results list based on the first query, the second query, and the third query (see at least Zhu figs.1, 5, 7, 9, c2:5-10 “generate a set of results responsive to the initial search query and refinements,” c7:60-67 “one or more language models may be utilized to weight the search server/engine 410 to influence the results returned from the query”. Please note: see previous comment.).Claim 7. The system of claim 1, wherein the control circuit further: ● selects a product from the product results list (see at least Zhu c9:40-45 “results include a number of stuffed bears, which the user may browse through for selection,” c10:15-30 “suggestions may be based on previous search results …. The user may then select one of these options to be used as the refinement input”); ● generates, via the language model, a product bundle based on attributes associated with the product (see at least Zhu c10:15-30 “suggestions may be based on previous search results …. user may provide an input image for Vitamin-C …. the system may provide suggestions such as "Vitamin-C capsules" or "Vitamin-C 300 grams" or "Brand A Vitamin-C",” c8:60-c9:10 “initial query 502 corresponds to an image, shown as a teddy bear. … one or more features of the initial query 502 may include ears, a snout, paws, fur, etc. … a set of search results with only the initial query 502 may lead to a mixture of relevant and irrelevant results … a refinement or additional query 504 … in this example is "bear" presented as a search string in order to try and limit results to not only the features of a bear, but to specifically include bears”); and ● causes the product bundle to be displayed with the product on the user interface (see at least Zhu figs. 6, c9:40-45 “results include a number of stuffed bears, which the user may browse through for selection”).Claim 8. The system of claim 7, wherein the product bundle is further generated based on the first query (see at least Zhu c8:60-c9:10 “initial query 502 corresponds to an image, shown as a teddy bear. … one or more features of the initial query 502 may include ears, a snout, paws, fur, etc. … a set of search results with only the initial query 502 may lead to a mixture of relevant and irrelevant results,” c9:40-45 “results include a number of stuffed bears, which the user may browse through for selection,” c10:15-30 “suggestions may be based on previous search results …. The user may then select one of these options to be used as the refinement input”).Claim 9. The system of claim 7, wherein the control circuit further: ● updates the product bundle, via the language model, based on the first query and the second query in response to receiving the second query (see at least Zhu figs.1, 5, 7, 9, c2:5-10 “generate a set of results responsive to the initial search query and refinements,” c7:60-67 “one or more language models may be utilized to weight the search server/engine 410 to influence the results returned from the query”).Claim 10. The system of claim 1, wherein the control circuit: ● generates, via the language model, a suggested search term based on the first query and the product database (see at least Zhu c8:60-c9:10 “initial query 502 corresponds to an image, shown as a teddy bear. … one or more features of the initial query 502 may include ears, a snout, paws, fur, etc. … a refinement or additional query 504 … in this example is "bear" presented as a search string in order to try and limit results,” c10:15-30 “suggestions may be based on previous search results …. user may provide an input image for Vitamin-C …. the system may provide suggestions such as "Vitamin-C capsules" or "Vitamin-C 300 grams" or "Brand A Vitamin-C"”); and ● updates, via the language model, the suggested search term based on the first query and the second query (see at least c10:15-40 “Embodiments may also provide suggestions for further refinements. … system may provide suggestions for further refinements. It should be appreciated that these suggestions may be based on previous search results, popular search results”); ● wherein the suggested search term is related to the first query or to the first query and the second query (see at least Zhu c8:60-c9:10 “initial query 502 corresponds to an image, shown as a teddy bear. … one or more features of the initial query 502 may include ears, a snout, paws, fur, etc. … a refinement or additional query 504 … in this example is "bear" presented as a search string in order to try and limit results,” c10:15-30 “suggestions may be based on previous search results …. user may provide an input image for Vitamin-C …. the system may provide suggestions such as "Vitamin-C capsules" or "Vitamin-C 300 grams" or "Brand A Vitamin-C"”. Please note: see previous comment concerning alternative limitations. It is noted that the second alternative here incorporates the first, so the broader limitation is all that is required to anticipate. Nonetheless both are anticipated as the reference describes suggested search terms beyond an initial search and refinement.).Claim 11. The system of claim 10, wherein the control circuit: ● updates, via the language model, the product results list based on a selected suggested search term (see at least Zhu c10:25-40 “user may then select one of these options to be used as the refinement input for processing via one or more trained models to generate a combined feature vector. … FIG. 6A illustrates an example environment 600 providing a set of search results,” c11:5-15 “In this example, rather than having the user type in the second input 622, the user may select one of a number of popular inputs or recommended inputs. In this case, the user has selected the "Vegan Capsules" option, as shown by the shading, and therefore the updated search results 624 are provided based on that selection”).Claim 12. The system of claim 1, wherein the control circuit further: ● updates the language model based on interaction data received via the user interface (see at least Zhu c6:40-50 “momentum updates and distillation may be utilized by the momentum encoders for different contrastive losses 316 between the encoders 302A, 302B, 304. It should be appreciated that momentum is provided as one example for training”).Claim 13. The system of claim 12, wherein the interaction data comprises at least one of selected products from the product results list, selected suggested search terms, or selected product bundles (see at least c11:10-15 “In this case, the user has selected the "Vegan Capsules" option, as shown by the shading, and therefore the updated search results 624 are provided based on that selection”).Pertaining to method claims 14-20
Rejection of claims 14-20 is based on the same rationale noted above.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Zhu et al. (Patent No.: US 12,210,516 B1) in view of Barbany et al. (nonpatent literature identified as item U on the attached form PTO-892).
Zhu teaches all of the above as noted. It teaches a) a system for providing product search results, b) providing a search interface to a plurality of user devices, c) a multimodal search interface, and d) generating product records as search results via a language model, but does not explicitly disclose wherein the language model comprises a large language model.
Barbany also teaches a) a system for providing product search results, b) providing a search interface to a plurality of user devices, c) a multimodal search interface, and d) generating product records as search results via a language model, and further discloses pertaining to
Claim 4. The system of claim 1, wherein the language model comprises a large language model (see at least Barbany abstract “we propose a novel search interface integrating Large Language Models (LLMs),” p.2 c1¶1 “interactive multimodal search solution leveraging recent advances in LLMs and vision-language models that can understand complex text queries”).
Therefore it would have been obvious to one of ordinary skill in the art at the time of invention (for pre-AIA applications) or filing (for applications filed under the AIA ) to modify the method of Zhu to include wherein the language model comprises a large language model, as taught by Barbany since the claimed invention is merely a combination of old elements and in the combination each element merely would have performed the same function as it did separately. One of ordinary skill in the art would have recognized that the results of the combination were predictable and would result in an improvement. This is because the level of ordinary skill in the art demonstrated by the references applied shows the ability to incorporate such features even from a variety of technical fields into methods and systems implemented using similar technological structures (i.e., generic computer and/or network hardware such as processors, servers, etc.). In this case the areas of technical endeavor are nonetheless similar and overlapping.
Applicant has not disclosed that the added feature solves any stated problem or is for any particular purpose beyond the performance of the functions they performed separately and since each element and its function are shown in the prior art the difference between the claimed subject matter and the prior art rests not on any individual element or function but in the very combination itself. It would therefore have been an obvious matter of design choice to include the feature from Barbany in the method of Zhu. Furthermore the combination solved no long felt need. Incorporating cumulative known features is additionally obvious to one of ordinary skill in the art because doing so increases commercial use of a method by attracting users that previously might have chosen between one of the previously known methods.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
● Dagan et al., Patent No.: US 12,265,582 B2: teaches query modality recommendation that recommends a modality based on search results from a first query with a first modality by comparing a first search performance of the first query modality in historical search queries to a second search performance of the second query modality in the historical search queries. Modality could be textual or image. "Many search systems allow users to submit search queries using different query modalities." c1:5-10.
● Agrawal et al., Patent No.: US 11,157,981 B2: teaches representing in an embedding space associated with an ecommerce website, images and items of text related to bids, mapping a query to the embedding space based on a comparison of features of the query to the images and the items of text, and selecting a bidder based on the mapping.
● Ehsani et al., Pub. No.: US 2013/0226892 A1: teaches capturing multi-modal interaction data wherein a user interface processes the interaction data and communicates with a user to establish a desirable query interpretation wherein a multimodal dialog interface can receive natural speech and other input modalities to perform a faceted search. A faceted search accepts a query with several search terms and refines or alters the search, possibly iteratively, and a dialog interface module can obtain an audio signal and another signal of a different modality, from the interfaces of the electronic device.
● Dykstra-Erickson et al., Patent No.: US 10,795,528 B2: teaches a task assistant and an interface for application that includes receiving input from a user through multimodal input including a plurality of speech input, typing input, and touch input, interpreting the input, and providing a formatted query to the application, receiving data from the application in response to the query, and providing a response to the user through multimodal output including a plurality of: speech output, text output, non-speech audio output, haptic output, and visual non-text output, and teaches filtering/updating search results with added search term, and removal of a filter term by verbal instruction. c12:39-c13:5
● Johnston et al., Pub. No.: US 2009/0089251 A1: teaches searching a content database using a first query parameter provided by a user via a first input modality and a second query parameter provided by the user via a second input modality. A multidimensional query is generated where the query is indicative of the first and second query parameters. The query is applied to the database to retrieve records of matching content, and the user then refines the results using additional multimodal queries.
● Johnson et al., Patent No.: US 6,807,529 B2: teaches multimodal communication sessions through differing user agent programs on one or more devices, for example an agent program communicating in voice mode, such as a voice browser in a voice gateway synchronized with an agent program operating in a graphical browser on a mobile device.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ADAM LEVINE whose telephone number is (571)272-8122. The examiner can normally be reached Monday - Thursday 9am-7:30pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Marissa Thein can be reached at 571.272.6764. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ADAM L LEVINE/Primary Examiner, Art Unit 3689 August 18, 2026
Prosecution Insights