DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
Applicant has not claimed priority to another application. Application 18/243,600 was filed 9/7/2023.
Information Disclosure Statement
No IDS has been submitted.
Status of Claims
Applicant’s amended claims, filed 3/19/2026, have been entered. Claims 1-4, 8-11, and 15-18 have been amended. Claims 1-20 are currently pending in this application and have been examined.
Indication of Subject Matter Overcoming Prior Art
Claims 1-20 are novel in view of the prior art and would be allowable if rewritten to overcome the claim rejection(s) under 35 U.S.C. 101 set forth in this Office Action.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim(s) recite(s) an abstract idea. This judicial exception is not integrated into a practical application. The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Under Step 1 of the Alice/Mayo test the claims are directed to statutory categories. Specifically, the method, as claimed in claims 1-7, are directed to a process, the non-transitory computer readable medium storage, as claimed in claims 8-14, are directed to an article of manufacture, and the computer program product, as claimed in claims 15-20, are directed to a machine, (see MPEP 2106.03).
Under Step 2A (prong 1), claim 1, taken as representative, recites at least the following limitations (emphasis added) that recite an abstract idea:
receiving an image for item information analysis, the image including image regions corresponding to unknown items in the image;
identifying a context of the image;
providing the context to an image-item analysis module;
generating, based on the context of the image, an item search space including a set of possible items in the image, wherein generating the item search space comprises limiting the item search space based on the context to reduce the set of possible items to apply the item information analysis;
identifying a set of detected items as a subset of the set of possible items in the image based on the image regions;
obtaining positional relationships of the set of detected items, wherein the positional relationships include relative position information related to the set of detected items, centers of mass for the set of detected items, and corresponding positional vectors, wherein the corresponding positional vectors are associated with the image regions and the centers of mass for the set of detected items;
identifying, for a detected item in the set of detected items, item information for the detected item, wherein the item information describes characteristics of the detected item including at least a portion of the positional relationships;
generating, by a model, a prompt for a language model of a content presentation module, wherein the model is to generate the prompt according to information obtained associated with the set of detected items, wherein the prompt comprises a predefined format associated with the set of detected items, wherein the predefined format comprises a text-based question and the item information;
providing a query to the language model, wherein the query comprises the prompt wherein the language model is to present an output including a size to improve visual assistance; and
sending the output of the language model for displaying according to the size.
These limitations recite certain methods of organizing human activity, such as performing commercial interactions (see MPEP 2106.04(a)(2)(II)). Certain methods of organizing human activity are defined by MPEP 2106.04 as including “fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions).” In this case, the abstract ideas recited in representative claim 1 are certain methods of organizing human activity because identifying items in an image and providing output (i.e., a recommendation) is a commercial or legal interaction because it is a advertising, marketing or sales activity, or business relations. Thus, claim 1 recites an abstract idea.
Independent claims 8 and 15 recite the same abstract idea as recited in independent claim 1. As such, the analysis under Step 2A, Prong 1 is the same for independent claims 8 and 15 as described above for independent claim 1.
Under Step 2A (prong 2), if it is determined that the claims recite a judicial exception, it is then necessary to evaluate whether the claims recite additional elements that integrate the judicial exception into a practical application of that exception (see MPEP 2106.04). As stated in the MPEP, when “an additional element merely recites the words ‘apply it (or an equivalent) with the judicial exception, or merely uses a computer as a tool to perform an abstract idea,” the judicial exception has not been integrated into a practical application. In this case, representative claim 1 includes additional elements such as (additional elements are bolded):
A method performed at a computer system comprising a processor and a computer-readable medium, the method comprising:
receiving an image for item information analysis, the image including image regions corresponding to unknown items in the image;
identifying a context of the image;
providing the context to an image-item analysis module;
generating, based on the context of the image, an item search space including a set of possible items in the image, wherein generating the item search space comprises limiting the item search space based on the context to reduce the set of possible items to apply the item information analysis;
identifying a set of detected items as a subset of the set of possible items in the image based on the image regions;
obtaining positional relationships of the set of detected items, wherein the positional relationships include relative position information related to the set of detected items, centers of mass for the set of detected items, and corresponding positional vectors, wherein the corresponding positional vectors are associated with the image regions and the centers of mass for the set of detected items;
identifying, for a detected item in the set of detected items, item information from an item database for the detected item, wherein the item information describes characteristics of the detected item including at least a portion of the positional relationships;
generating, by a neural network model, a prompt for a language model of a content presentation module, wherein the neural network model is trained to generate the prompt according to information obtained associated with the set of detected items, wherein the prompt comprises a predefined format associated with the set of detected items, wherein the predefined format comprises a text-based question and the item information;
providing a query to the language model, wherein the query comprises the prompt wherein the language model is trained to present an output including a size to improve visual assistance; and
sending the output of the language model to a user device for displaying according to the size.
Independent claims 8 and 15 recite the additional elements of “a non-transitory computer readable storage having instructions encoded thereon that, when executed by a processor, cause the processor to perform steps,” a “computer program product, comprising: a processor that executes instructions; and a non-transitory computer readable storage medium having instructions executable by the processor” in addition to the additional elements already addressed in the rejection for independent claim 1.
Although reciting these additional elements, taken alone or in combination these elements are not sufficient to integrate the abstract idea into a practical application. These additional elements merely amount to the general application of the abstract idea to a technical environment (“performed at a computer system comprising a processor and a computer-readable medium”, “from an item database”, by trained “a neural network model”, “to a user device”) and insignificant pre-and-post solution activity (receiving/obtaining information, providing/sending information). The specification makes clear the general-purpose nature of the technological environment. This is because the additional elements of claims 1, 8, and 15 are recited at a high level of generality (i.e., as generic computing hardware) such that they amount to nothing more than the mere instructions to implement or apply the abstract idea on generic computing hardware (or, merely uses a computer as a tool to perform the abstract idea) (see Fig. 1; paragraphs [0012], [0017], [0026], [0048]-[0052], [0055], [0058]-[0062] and [0052]). The specification indicates that while exemplary general-purpose systems may be specific for descriptive purposes, any elements capable of implementing the claimed invention are acceptable. That is, the technology used to implement the invention is not specific or integral to the claim. The description demonstrates that these additional elements are merely generic devices such as a generic computer. Further, the additional elements do no more than generally link the use of a judicial exception to a particular environment or field of use (such as the Internet or computing networks).
Therefore, considered both individually and as an ordered pair, the additional elements do no more than generally link the use of the abstract idea to a particular technological environment or field of use. That is, given the generality with which the additional elements are recited, the limitations do not implement the abstract idea with, or use the abstract idea in conjunction with, a particular machine or manufacture that is integral to the claim. Additionally, the claims do not reflect an improvement in the functioning of a computer, or an improvement to other technology or technical field, do not transform or reduction of a particular article to a different state or thing; and do not apply or use the abstract idea in some other meaningful way beyond generally linking the use of the abstract idea to a particular technology environment, such that the claim as a whole is more than a drafting effort designed to monopolize the abstract idea into a practical application, and is therefore “directed to” the abstract idea.
In addition to the above, the recited receiving/obtaining information, providing/sending information steps (even assuming arguendo they do not form part of the abstract idea, which the Examiner does not acquiesce), are at best little more than extra-solution activity (e.g., data gathering, presentation of data) that contributes nominally or insignificantly to the execution of the claimed system (see MPEP 2106.05(g)).
In view of the above, under Step 2A (prong 2), claims 1, 8, and 15 do not integrate the recited exception into a practical application.
Under Step 2B, examiners should evaluate additional elements individually and in combination to determine whether they provide an inventive concept (i.e., whether the additional elements amount to significantly more than the exception itself). In this case, the claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception.
Returning to representative claims 1, 8, and 15, taken individually or as a whole the additional elements of claims 1, 8, and 15 do not provide an inventive concept (i.e. they do not amount to “significantly more” than the exception itself). As discussed above with respect to the integration of the abstract idea into a practical application, the additional elements used to perform the claimed process amount to no more than the mere instructions to apply the exception using a generic computer and/or no more than a general link to a technological environment.
Furthermore, the additional elements fail to provide significantly more also because the claim simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception. For example, the additional elements of claims 1, 8, and 15 utilize operations the courts have held to be well-understood, routine, and conventional (see: MPEP 2106.05(d)(II)), including at least:
receiving or transmitting data over a network,
storing or retrieving information from memory,
presenting offers
Additionally, the Specification recites machine learning models, neural networks, and training machine learning models are well-understood, routine, and conventional activities previously known to the industry (see paragraphs [0048]-[0051]).
Even considered as an ordered combination (as a whole), the additional elements of claims 1, 8, and 15 do not add anything further than when they are considered individually.
In view of the above, representative claims 1, 8, and 15 do not provide an inventive concept (“significantly more”) under Step 2B, and is therefore ineligible for patenting.
Regarding claims 2-7, 9-14, and 16-20
Dependent claim(s) 2-7, 9-14, and 16-20, when analyzed as a whole, are held to be patent ineligible under 35 U.S.C. 101 because they do not add “significantly more” to the abstract idea. More specifically, dependent claim(s) 2-7, 9-14, and 16-20 merely further define the abstract limitations of claim(s) 1, 8, and 15 or provide further embellishments of the limitations recited in independent claim claim(s) 1, 8, and 15.
Claims 2-7, 9-14, and 16-20 set forth:
wherein identifying the context of the image comprises receiving a request from a user for delivered items, wherein the context includes the request and the set of possible items include a set of items from an order delivered to the user.
wherein identifying the context of the image comprises determining a replacement item for an unavailable item in an order, wherein the set of possible items include a set of stocked items in a warehouse at which the image was captured.
Wherein querying comprises prompting the language model with structured item information about the detected item and a request to analyze the structured item information.
wherein prompting the language model comprises requesting identification of a question a user may have about the detected item.
wherein the output of the language model describes a question, and the method further comprises: determining an answer to the question from the language model based on the question and the item information; and providing the answer to the user device for display to the user responsive to the user selecting the question.
wherein identifying the set of detected items comprises performing a nearest-neighbor search.
wherein the output of the language model describes a question, and wherein the instructions are further executable for: determining an answer to the question from the language model based on the question and the item information; and providing the answer to the user device for display to the user responsive to the user selecting the question.
Such recitations merely embellish the abstract idea of identifying items in an image and providing output (i.e., a recommendation). The claims do not set forth any further additional limitations, and therefore such abstract embellishments are applied to the additional limitations recited in claim(s) 1, 8, and 15, which do no more than generally link the use of the abstract idea to a particular technological environment, do not integrate the abstract idea into a practical application, and do not provide an inventive concept. Accordingly, the claims do not confer eligibility on the claimed invention and is ineligible for similar reasons to claim(s) 1, 8, and 15.
Thus, dependent claims 2-7, 9-14, and 16-20 are ineligible.
Response to Arguments
Applicant’s arguments, on pages 11-13 of the Remarks filed 3/19/2026, with respect to the previous 35 USC §101 rejections have been fully considered but they are not persuasive. Applicant argues the amended claims cover a practical application of computer vision and machine learning techniques to solve a specific technological problem: inability to determine information from an image, such as users with vision impairments. Examiner respectfully disagrees.
While Applicant asserts that the features in the claims provide an improvement by citing paragraph [0001] of the Specification reciting “Customers may need to know additional information beyond what is available in the picture to determine whether to order the item. This problem may be compounded by users who cannot readily determine information from an image, such as users with vision difficulties or who may be partially or fully blind,” these arguments are directed to the improvement of the abstract idea and not a technological improvement.
Applicant further asserts on pages 11-13 of the Remarks that the present claims are rooted in computer technology and recite a technical solution. Examiner respectfully disagrees. The specification should disclose sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement, and the claim itself must reflect the improvement in technology (see MPEP 2106.04(a)(I)). As currently claimed, the claims do not reflect an improvement in technology as the argued claim limitations are broadly claimed and the argued claim amendments are directed to the abstract idea (i.e., providing a specific data flow, performing segmentation, embedding-based matching, and structured prompt constructing, incorporation relation position data and item attributes to directed processing of a language model, and obtaining position relations of the set of detected items, wherein the positional relationships include relative position information related to the set of detected items, centers of mass for the set of detected items, and corresponding positional vectors are, as currently claimed, directed to the abstract idea). Merely performing these abstract ideas via a “online concierge system” do not add meaningful limitations to the abstract idea beyond generally linking the invention to a particular technological environment, that is, implementation via computers/software (see MPEP 2106.04).
While Applicant cites to paragraphs [0071] and [0074] of the Specification, Examiner notes these cited paragraphs do not describe a technological improvement. During patent examination, the pending claims must be "given their broadest reasonable interpretation consistent with the specification" (see MPEP 2111). As currently described in the Specification and as currently claimed, analyzing an image to detect items in the image that match the possible items in the item search space, applying image segmentation algorithms to identify separate items, determining the possible items in the image, and information about relative relationships and positions of the items in the image in order to alter the image search space with context narrowing and structuring prompts with precise item information and spatial context are directed to the abstract idea and are not describing a technological improvement. The specification does not disclose sufficient details such that one of ordinary skill in the art would recognize the claimed invention as providing an improvement to the technology and the claim itself does not reflect an improvement in technology (see MPEP 2106.04(a)(I)).
Therefore, the Examiner maintains the claims are not significantly more than the abstract idea.
Accordingly, the 35 USC §101 rejections of the claims are maintained.
Applicant’s arguments, on pages 13-15 of the Remarks filed 3/19/2026, with respect to the 35 USC §102 and 35 USC §103 rejections have been fully considered and are persuasive in view of the claim amendments. Accordingly, the previous 35 USC §102 and 35 USC §103 rejections of the claims have been withdrawn.
Examiner’s Comment
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Reference A of the Notice of References Cited Shiraishi et al. (US 2023/0186271 A1) discloses detecting an object from an image based on generated positional information including center of mass, generating product kind-related information that determines a product kind for each detected object, and outputs a product recognition result for each extracted group.
Conclusion
Applicant's amendment necessitated the new ground(s) of rejection presented in this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to LINDSEY B SMITH whose telephone number is (571)272-0519. The examiner can normally be reached Monday - Friday 9-6 EST.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jeff Smith can be reached at 571-272-6763. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
LINDSEY B. SMITH
Examiner
Art Unit 3688
/LINDSEY B SMITH/Examiner, Art Unit 3688
/Jeffrey A. Smith/Supervisory Patent Examiner, Art Unit 3688
Prosecution Insights