DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Objections
Claim 4 is objected to because of the following informalities: line 4 “from the large language model and .” is incomplete. Appropriate correction is required.
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claim(s) 1-7 are rejected under 35 U.S.C. 102(a)(2) as being anticipated by He (US 20240378656 A1).
Regarding claim 1, He teaches a method of controlling access to image data collected by a user device, comprising:
capturing image data; ([0021]: the picker client device 110 captures one or more images of the item and determines the item identifier for the item based on the images.)
generating a plurality of tokens based on the image data; ([0031]: The model serving system receives a request including input data (e.g., text data, audio data, image data, or video data) and encodes the input data into a set of input tokens.)
selecting a first subset of the plurality of tokens; ([0031]: For an example query processing task, the language model may receive a sequence of input tokens (e.g. first subset) that represent a query.)
generating a first query based on the first subset of the plurality of tokens; and ([0031]: For example, a token may correspond to a word, a punctuation symbol, a space, a phrase, a paragraph, and the like. For an example query processing task, the language model may receive a sequence of input tokens that represent a query and generate a sequence of output tokens that represent a response to the query.)
transmitting the first query to a large language model. ([0033]: the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks.)
Regarding claim 2, He teaches the method of claim 1.
He teaches obtaining a prompt from a user and where the first subset of the plurality of tokens are selected based on the prompt. ([0043]: The interface system 160 constructs one or more prompts for input to the model serving system 150. A prompt may include the query of the user and context obtained from the structured index of the external data. In one instance, the context in the prompt includes portions of the structured indices as contextual information for the query. [0065]: a prompt of an image provided by a picker client device 110 with instructions to identify the out-of-stock target item and potential replacement items may include specifications for specific key value pairs as output.)
Regarding claim 3, He teaches the method of claim 2.
He teaches further comprising transmitting the first query to the large language model and receiving a response to the prompt from the large language model. ([0031]: For an example query processing task, the language model may receive a sequence of input tokens that represent a query and generate a sequence of output tokens that represent a response to the query. [0033]: the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks.)
Regarding claim 4, He teaches the method of claim 3.
He teaches further comprising transmitting a second query to the large language model, where the second query is generated from a second subset of the plurality of tokens that are selected based on the response from the large language model and . ([0073]: In some embodiments, when the replacement suggestion module receives a response with an explanation indicating a failure (e.g., response), the replacement suggestion module may request from picker client device information for a new prompt (e.g., second query) as input for the LLM. For example, the picker may choose to take a new image or manually enter the information instead of sending an image to the LLM.)
Regarding claim 5, He teaches the method of claim 1.
He teaches where the image data is captured according to user-defined access control. ([0021]: the picker client device 110 captures one or more images of the item and determines the item identifier for the item based on the images. [0037].)
Regarding claim 6, He teaches the method of claim 5.
He teaches where the image data is captured based on a request from the large language model. ([0073]: In some embodiments, when the replacement suggestion module receives a response with an explanation indicating a failure, the replacement suggestion module may request from picker client device information for a new prompt as input for the LLM. For example, the picker may choose to take a new image or manually enter the information instead of sending an image to the LLM.)
Regarding claim 7, He teaches the method of claim 5.
He teaches where the first subset of the plurality of tokens are selected based on a request from the large language model. ([0031]: For an example query processing task, the language model may receive a sequence of input tokens (e.g. first subset) that represent a query. [0037]: the online system 140 prepares a prompt for input to the model serving system 150. The prompt may include an image taken by a picker of an out-of-stock target item. The image may further include items on nearby shelves.)
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 8-20 are rejected under 35 U.S.C. 103 as being unpatentable over He (US 20240378656 A1) in view of Lv (CN 116595977 A).
Regarding claim 8, He teaches the method of claim 1.
He does not explicitly disclose where the first subset of the plurality of tokens are selected according to user-defined access control.
However, Lv teaches where the first subset of the plurality of tokens are selected according to user-defined access control. (Page 2: step one - five. step one: the user uses the user-defined input assembly system to complete data input, the user inputs (e.g., first subset of tokens) the content needing to interact with the large language model into the user-defined input assembly.step four: if the second step and the third step do not find sensitive data, the data of the custom input assembly can be provided to the large language model by default. step five: if the system finds sensitive data in the second step and the third step, the user is prompted to authorize before the data is improved to the large language model, after the user completes authorization confirmation, a user record is added to the risk operation record system, and meanwhile the local data is transmitted to the large data model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He to include above limitations. One would have been motivated to do so because in the process of talking with the large models, users tend to leak core data of enterprises, individuals or other people to manufacturers providing large model generation technologies in an unintentional manner. The patent mainly solves the problem that before a user submits data to a large language model, a layer of PII (personal identity information) real-time detection technology is additionally added, wind control recording is carried out, and leakage of user information sensitive data in the process of using the large language model is prevented. As taught by Lv, Page 1 Background.
Regarding claim 9, He teaches the method of claim 1.
He does not explicitly disclose where the first query is generated according to user-defined access control.
However, Lv teaches where the first query is generated according to user-defined access control. (Page 2: step one - five. step one: the user uses the user-defined input assembly system to complete data input, the user inputs (e.g., first query) the content needing to interact with the large language model into the user-defined input assembly. step four: if the second step and the third step do not find sensitive data, the data of the custom input assembly can be provided to the large language model by default. step five: if the system finds sensitive data in the second step and the third step, the user is prompted to authorize before the data is improved to the large language model, after the user completes authorization confirmation, a user record is added to the risk operation record system, and meanwhile the local data is transmitted to the large data model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He to include above limitations. One would have been motivated to do so because in the process of talking with the large models, users tend to leak core data of enterprises, individuals or other people to manufacturers providing large model generation technologies in an unintentional manner. The patent mainly solves the problem that before a user submits data to a large language model, a layer of PII (personal identity information) real-time detection technology is additionally added, wind control recording is carried out, and leakage of user information sensitive data in the process of using the large language model is prevented. As taught by Lv, Page 1 Background.
Regarding claim 10, He teaches an apparatus, comprising:
a sensor; a machine learning logic; a processor; and a non-transitory computer-readable medium comprising instructions that, when executed by the processor, cause the processor to: (Fig. 1A and 1B)
capture first data via the sensor; ([0021]: the picker client device 110 captures one or more images of the item and determines the item identifier for the item based on the images.)
generate a plurality of tokens based on the first data; and ([0031]: The model serving system receives a request including input data (e.g., text data, audio data, image data, or video data) and encodes the input data into a set of input tokens.)
He does not explicitly disclose select a first subset of the plurality of tokens based on user-defined access control.
However, Lv teaches select a first subset of the plurality of tokens based on user-defined access control. (Page 2: step one - five. step one: the user uses the user-defined input assembly system to complete data input, the user inputs (e.g., first subset of tokens) the content needing to interact with the large language model into the user-defined input assembly. step four: if the second step and the third step do not find sensitive data, the data of the custom input assembly can be provided to the large language model by default. step five: if the system finds sensitive data in the second step and the third step, the user is prompted to authorize before the data is improved to the large language model, after the user completes authorization confirmation, a user record is added to the risk operation record system, and meanwhile the local data is transmitted to the large data model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He to include above limitations. One would have been motivated to do so because in the process of talking with the large models, users tend to leak core data of enterprises, individuals or other people to manufacturers providing large model generation technologies in an unintentional manner. The patent mainly solves the problem that before a user submits data to a large language model, a layer of PII (personal identity information) real-time detection technology is additionally added, wind control recording is carried out, and leakage of user information sensitive data in the process of using the large language model is prevented. As taught by Lv, Page 1 Background.
Regarding claim 11, He and Lv teach the apparatus of claim 10.
He teaches further comprising a network interface, and where the instructions further cause the processor to generate a first query based on the first subset of the plurality of tokens and transmit the first query to a large language model via the network interface. (Fig. 1A and 1B. [0031]: For an example query processing task, the language model may receive a sequence of input tokens that represent a query and generate a sequence of output tokens that represent a response to the query. [0033]: the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks.)
Regarding claim 12, He and Lv teach the apparatus of claim 10.
He teaches further comprising a network interface, and where the instructions further cause the processor to capture the first data via the sensor based on a query received from a large language model via the network interface. ([0031]: For an example query processing task, the language model may receive a sequence of input tokens that represent a query. [0037]: the online system 140 prepares a prompt for input to the model serving system 150. The prompt may include an image taken by a picker (e.g. capture the first data) of an out-of-stock target item. The image may further include items on nearby shelves.)
Regarding claim 13, He and Lv teach the apparatus of claim 10.
He teaches where the instructions further cause the processor to store the first subset of the plurality of tokens for future reference. ([0073]: the image that failed and the manually information may be used to further train the LLM for future use. [0084]: if the customer selects an item from the image, that selection may be used to update and train the multi-modality LLM API 340 for future predictions.)
Regarding claim 14, He and Lv teach the apparatus of claim 13.
He teaches further comprising a network interface, and where the instructions further cause the processor to select a second subset of the plurality of tokens from the first subset based on a user prompt, generate a first query based on the second subset, and transmit the first query to a large language model via the network interface. ([0066]: the picker client device 110 may further provide, to the replacement suggestion module 225, input signals from the user, for example, text, voice, etc. The picker user (e.g., picker) may include additional instructions/comments in the input signal which may be used to construct the prompt with the image. The user may input a customized request for the customer, e.g., “replacement with organic food only.” The replacement suggestion module 225 may extract key values pairs from the input signal and construct the prompt. [0031]: For an example query processing task, the language model may receive a sequence of input tokens that represent a query and generate a sequence of output tokens that represent a response to the query. [0033]: the language models are large language models (LLMs) that are trained on a large corpus of training data to generate outputs for the NLP tasks.)
Regarding claim 15, He teaches an apparatus, comprising:
a network interface configured to communicate with user devices via a first API and communicate with external network resources via a second API; (Fig. 1B. Fig. 3. [0019]: the picker client device executes a client application that uses an application programming interface (API) to communicate with the online system. [0077]: The online system 140 uploads the image 310 to a multi-modality LLM API 340 and retrieves, using multi-modality LLM API 340, a list of one or more recommended replacement items.)
a processor; and a non-transitory computer-readable medium comprising instructions that, when executed by the processor, cause the processor to: (Fig. 1A and 1B)
in response to a first request for first user context from a first network resource via the second API, (Fig. 3. [0031]: For example, a token may correspond to a word, a punctuation symbol, a space, a phrase, a paragraph, and the like. For an example query processing task, the language model may receive a sequence of input tokens that represent a query and generate a sequence of output tokens that represent a response to the query. [0037].)
provide the first user context to the first network resource. ([0077]: The online system 140 uploads the image 310 to a multi-modality LLM API 340 and retrieves, using multi-modality LLM API 340, a list of one or more recommended replacement items.)
He does not explicitly disclose authorize the first network resource based on user-defined access control; and provide the first user context to the first network resource when successfully authorized.
However, Lv teaches authorize the first network resource based on user-defined access control; and provide the first user context to the first network resource when successfully authorized. (Page 2: step one - five. step one: the user uses the user-defined input assembly system to complete data input, the user inputs (e.g., first subset of tokens) the content needing to interact with the large language model into the user-defined input assembly. step four: if the second step and the third step do not find sensitive data, the data of the custom input assembly can be provided to the large language model by default. step five: if the system finds sensitive data in the second step and the third step, the user is prompted to authorize before the data is improved to the large language model, after the user completes authorization confirmation, a user record is added to the risk operation record system, and meanwhile the local data is transmitted to the large data model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He to include above limitations. One would have been motivated to do so because in the process of talking with the large models, users tend to leak core data of enterprises, individuals or other people to manufacturers providing large model generation technologies in an unintentional manner. The patent mainly solves the problem that before a user submits data to a large language model, a layer of PII (personal identity information) real-time detection technology is additionally added, wind control recording is carried out, and leakage of user information sensitive data in the process of using the large language model is prevented. As taught by Lv, Page 1 Background.
Regarding claim 16, He and Lv teach the apparatus of claim 15.
He teaches where the instructions further cause the processor to cause a first user device to capture edge data and where the first user context is based on the edge data. ([0021]: the picker client device 110 captures one or more images of the item and determines the item identifier for the item based on the images.)
Regarding claim 17, He and Lv teach the apparatus of claim 15.
He teaches where the first user context is based on accumulated user context. ([0031]: For example, a token may correspond to a word, a punctuation symbol, a space, a phrase, a paragraph, and the like. For an example query processing task, the language model may receive a sequence of input tokens that represent a query and generate a sequence of output tokens that represent a response to the query.)
Regarding claim 18, He and Lv teach the apparatus of claim 15.
He teaches in response to a second request to modify a second user context from a first user device via the first API, modify the second user context. ([0066]: the picker client device 110 may further provide, to the replacement suggestion module 225, input signals from the user, for example, text, voice, etc. The picker user (e.g., picker) may include additional instructions/comments in the input signal which may be used to construct the prompt with the image. The user may input a customized request for the customer, e.g., “replacement with organic food only.” The replacement suggestion module 225 may extract key values pairs from the input signal and construct the prompt.)
He does not explicitly disclose authenticate the first user device; and modify the second user context when successfully authenticated.
However, Lv teaches authenticate the first user device; and modify the second user context when successfully authenticated. (Page 2: step one - five. step one: the user uses the user-defined input assembly system to complete data input, the user inputs (e.g., first subset of tokens) the content needing to interact with the large language model into the user-defined input assembly. step four: if the second step and the third step do not find sensitive data, the data of the custom input assembly can be provided to the large language model by default. step five: if the system finds sensitive data in the second step and the third step, the user is prompted to authorize before the data is improved to the large language model, after the user completes authorization confirmation, a user record is added to the risk operation record system, and meanwhile the local data is transmitted to the large data model.)
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to modify He to include above limitations. One would have been motivated to do so because in the process of talking with the large models, users tend to leak core data of enterprises, individuals or other people to manufacturers providing large model generation technologies in an unintentional manner. The patent mainly solves the problem that before a user submits data to a large language model, a layer of PII (personal identity information) real-time detection technology is additionally added, wind control recording is carried out, and leakage of user information sensitive data in the process of using the large language model is prevented. As taught by Lv, Page 1 Background.
Regarding claim 19, He and Lv teach the apparatus of claim 15.
He teaches where the instructions further cause the processor to cause a first user device to notify a user of the first request for the first user context from the first network resource. (Fig. 6. [0039]: The online system obtains the response. The response may include a message to the user written to match a provided template. In some embodiments, the information from the response is provided to a Picker API and Picker Customer Chat API, so that the user may select a replacement item.)
Regarding claim 20, He and Lv teach the apparatus of claim 19.
Lv teaches where the first network resource is only authorized for a specific query of a large language model. (step five: if the system finds sensitive data in the second step and the third step, the user is prompted to authorize before the data is improved to the large language model, after the user completes authorization confirmation, a user record is added to the risk operation record system, and meanwhile the local data is transmitted to the large data model.)
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZI YE whose telephone number is (571)270-1039. The examiner can normally be reached Monday - Friday, 8:00am - 4:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Emmanuel Moise can be reached at 5712723865. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZI YE/Primary Examiner, Art Unit 2455