DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This action is in response to the applicant’s response filed 29 May 2025, which is in response to the USPTO office action mailed 11 March 2025. Claim 8 is amended. Claims 1-20 are currently pending.
Response to Arguments
With respect to the 35 USC §103 rejections of claims 1-20, the applicant’s arguments have been fully considered but have not been deemed persuasive. Firstly, the applicant argues “Feng teaches a CRF- or FSM-based query parser that assigns to each token ‘class-level multi-gram language model-based features’ harvested from ‘large-scale class-based logs’ (e.g., GEO, ST), with each ‘class’ operating as a separate, parallel feature category. The term ‘class’ and ‘multi-gram’ ultimately refer to a flat, i.e. non-hierarchical structure” (remarks pg. 9). Respectfully, this argument is not persuasive.
In particular, Feng teaches “a query parser derivation computing device using a rule-based approach using a finite state machine ("FSM") and a statistical approach using a statistical sequential labeling model, such as a conditional random field ("CRF") model. For the FSM approach, phrase level grammars are used and grammar composition is applied by a query parser derivation computing device, to build a large-scale query parser. For the CRF based parser, the query parser derivation computing device, learns class transition features from a labeled set and extracts high-level language models as state features” (Feng, [0026]).
To clarify the rejection, the examiner interprets the FSM and CRF model taught by Feng reads on a hierarchical structure because a FSM is a model in which one state transitions to another state based on inputs to the FSM. In other words, state transitions are a sequence of dependent states. Similarly, the CRF model labels sequential data, i.e. data forming or following in a logical order or sequence. Accordingly, this argument is not persuasive.
Secondly, the applicant argues “Nothing in Feng suggests that one class builds upon, nests within, or otherwise depends on another; the classes (‘GEO,’ ‘ST,’ etc.) are applied side-by-side to the same token stream. Accordingly, the word ‘level’ in Feng refers only to the n-gram length and class tag of a single feature, not to any tiered or parent-child structure” (remarks pg. 9).
In response to the applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., classes or categories that builds upon, nests within, or otherwise depends on another in a tiered or parent child structure) are not recited in the rejected claims. Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. Accordingly, this argument is not persuasive.
Lastly, the applicant argues “Feng contains no teaching, hint, or suggestion of a ‘context shift.’ The omission of ‘context shift’, alone, renders Feng moot. However the simple fact is that Feng only discloses a CRF/FSM parser that evaluates every query with the same fixed feature set, without any hierarchical contextualization” (remarks pg. 9).
The applicant's arguments fail to comply with 37 CFR 1.111(b) because they amount to a general allegation that the claims define a patentable invention without specifically pointing out how the language of the claims patentably distinguishes them from the references. In particular, the claims as currently drafted do not include any details which may differentiate the “context shift” over the FSM/CRF disclosed by Feng. Accordingly, this argument is not persuasive.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 1, 3, 5, 8, 10-12 and 14-18 are rejected under 35 U.S.C. 103 as being unpatentable over Feng et al., US 2013/0031113 A1 (hereinafter “Feng”) in view of Dong et al., US 2024/0193445 A1 (hereinafter “Dong”).
Claim 1: Feng teaches a method for real-time search comprising:
configuring a language model engine with a context shift using a plurality of specific contextualization objects within a hierarchy dependent from a general contextualization object (Feng, [0020] note a multi-gram language model contains features of n tokens (words/symbols) in length, where n is an integer, [0026] note a query parser derivation computing device using a rule-based approach using a finite state machine ("FSM") and a statistical approach using a statistical sequential labeling model, such as a conditional random field ("CRF") model. For the FSM approach, phrase level grammars are used and grammar composition is applied by a query parser derivation computing device, to build a large-scale query parser);
receiving, at a natural language processing (NLP) engine, an input message (Feng, [0005] note queries using natural language, [0042] note a query is received by the query parser. i.e., "find the best bar in Los Angeles, Calif."; i.e. a natural language query, [Fig. 5] note 104, [0046] note query parser system 100 may receive a query 104);
forwarding the input message from the collaboration platform to the engine (Feng, [0024] note a query parser derivation computing device, leverages as features high-level class-based language models, [Fig. 5], [0046] note query parser system 100 may receive a query 104, e.g., over the internet 106 and provide the query 104 to the query parser derivation computing device 102; i.e. providing the query reads on forwarding the input message);
determining, at the engine, that the input message includes an unstructured query corresponding to one of the specific contextualization objects based on the general context shift and the unstructured query (Feng, [Fig. 6] note 154, [0047] note FIG. 6 there is shown in block diagram format an example of a query parser derivation process 150 performed by the query parser derivation computing device 102 of FIG. 5… process 150 may receive a query in block 152 and determine if the query is structured or unstructured in block 154);
the number of tokens in the determining being less than if the determining is based on a non-hierarchical contextualization object (Feng, [0020] note The learned model can use class-level multi-gram language model-based features. Such features of multi-gram language models, harvested from tokens contained in a structured queries log, can insulate the model from surface-level tokens, [0064] note Parsing the input query may recognize both a geographic location token and a search term token. Phrase level grammar may be built using a context free grammar composition);
preparing, at the engine, a draft response message including a structured query to at least one of a plurality of search engines corresponding to the one of the specific contextualization objects (Feng, [Fig. 5], [Fig. 6], [0047] note If the query is unstructured the query may be delivered to the system shown in FIG. 5, including the query parser derivation computing device 102 to produce a query parser and parse the query to a structured query);
forwarding the draft response message from the engine to the collaboration platform; sending the structured query from the collaboration platform to a management engine for routing and processing by the at least one of a plurality of search engines (Feng, [Fig. 5], [Fig. 6] note 160, [0047] note parse the query to a structured query, which can then be provided to the search engine in the step of block 160);
receiving, at the collaboration platform, a response to the structured query, from the search engine via the management engine; and, generating an output message responsive to the input message including the draft response message that substitutes the response for the structured query (Feng, [0046] note the search engine may then used the parsed query to perform a search, e.g., in databases 120 to provide a query result 114, e.g., back through the internet 106 to the originator of the query 104; i.e. query results read on an output message).
Feng does not explicitly teach a large language model (LLM).
However, Dong teaches this (Dong, [0024] note a machine learning model(s)—such as a large language model (LLM)—may include a base model(s) and domain specific parts… The base model(s) may include layers that are trained using training data (general-purpose training data) associated with multiple domains. As described herein, a domain may include, but is not limited to, a financing domain, a travel domain, a communications domain, a computer science domain, an automotive domain, an electronics domain, a real estate domain, and/or any other type of domain, [0031] note a part may be trained to understand input data associated with different intents (e.g., booking travel, requesting information, interpreting information, etc.), different tasks (e.g., with regard to booking travel, booking a plane flight, booking a cruise, booking a hotel, etc.), and/or the like).
It would have been obvious to one of ordinary skill in the art at the effective filing date of the application to combine the multi-gram language models of Feng with the LLM including base models and domain specific parts of Dong according to known methods (i.e. using a large language model including base models and domain specific parts). Motivation for doing so is that by activating and/or deactivating domain specific parts of the machine learning model(s) based on the domains for which the machine learning model(s) is deployed, the machine learning model(s) of the current systems may be more accurate, require fewer computing resources, and/or have less latency when processing input data (Dong, [0006]).
Claim 3: Feng and Dong teach the method of claim 1 wherein the general contextualization object is based on travel searching and configures the LLM engine to classify the unstructured query into at least one of an air-search query, an air-policy query, a general query, an events query and a ground transportation itinerary search query; each of the queries corresponding to one or more of the search engines (Feng, [0060] note utilizing in the rule based finite state machine model a phrase level grammar to construct a phrase level grammar composition to form the query parser. The system and method disclosed may comprise utilizing in the statistics-based conditional random fields model class transition features learned from a labeled set of features and extracting at least one high-level language model as a state feature, thereby allowing at least one learned language model to be learned from at least one large-scale class-based log to improve coverage on an unknown query and parsing an unstructured local geographical web-based query in local domain by deriving a learned model parser to parse the query).
Claim 5: Feng and Dong teach the method of claim 3 wherein the real-time search is a travel query and the search engines are travel actor engines; the travel query includes a transportation-actor component and a hospitality-actor component and the transportation-actor component is respective to at least one travel actor engine and the hospitality-actor component is respective to another at least one travel actor engine (Dong, [0031] note a part may be trained to understand input data associated with different intents (e.g., booking travel, requesting information, interpreting information, etc.), different tasks (e.g., with regard to booking travel, booking a plane flight, booking a cruise, booking a hotel, etc.), and/or the like).
Claim 8: Feng and Dong teach the method of claim 3 wherein travel query requires a coordination between travel-actors such that the results are responsively filtered by the coordination (Dong, [0001] note Language models are used in many different applications, such as to schedule travel plans (e.g., booking arrangements for transportation and accommodations etc.), plan activities (e.g., making reservations, etc.), communicate with others (e.g., make phone calls, start video conferences, etc.), shop for items (e.g., purchase items from online marketplaces, etc.), and/or other use cases).
Claim 10: Feng and Dong teach the method of claim 3 wherein the travel query includes one or more travel-actors including: transportation-actors including airlines, rail services, bus lines and ferry lines; hospitality-actors including hotels, resorts and bed and breakfasts; for-hire ground-transportation actors including car-rentals, taxis and car sharing; and dining-actors including restaurants, bistros and bars (Dong, [0031] note a part may be trained to understand input data associated with different intents (e.g., booking travel, requesting information, interpreting information, etc.), different tasks (e.g., with regard to booking travel, booking a plane flight, booking a cruise, booking a hotel, etc.), and/or the like).
Claim 11: Feng and Dong teach the method of claim 1 wherein the input message and output message are incorporated into a collaboration tool executing on a collaboration platform that hosts the NLP engine (Feng, [0058] note the above disclosed subject matter may be useful for providing local search applications/services (including business, product and utility searches), e.g., with a focus on social media analysis or real-time searches).
Claim 12: Feng and Dong teach the method of claim 11 wherein the collaboration tool is a social media platform (Feng, [0058] note the above disclosed subject matter may be useful for providing local search applications/services (including business, product and utility searches), e.g., with a focus on social media analysis or real-time searches).
Claim 14: Feng teaches a collaboration platform including a real time network search function based on natural language processing queries; the platform including a processor and a memory; the processor executing programming instructions for:
configuring a language model engine with a context shift using a plurality of specific contextualization objects within a hierarchy dependent from a general contextualization object (Feng, [0020] note a multi-gram language model contains features of n tokens (words/symbols) in length, where n is an integer, [0026] note a query parser derivation computing device using a rule-based approach using a finite state machine ("FSM") and a statistical approach using a statistical sequential labeling model, such as a conditional random field ("CRF") model. For the FSM approach, phrase level grammars are used and grammar composition is applied by a query parser derivation computing device, to build a large-scale query parser);
receiving, at a natural language processing (NLP) engine, an input message (Feng, [0005] note queries using natural language, [0042] note a query is received by the query parser. i.e., "find the best bar in Los Angeles, Calif."; i.e. a natural language query, [Fig. 5] note 104, [0046] note query parser system 100 may receive a query 104);
forwarding the input message from the collaboration platform to the engine (Feng, [0024] note a query parser derivation computing device, leverages as features high-level class-based language models, [Fig. 5], [0046] note query parser system 100 may receive a query 104, e.g., over the internet 106 and provide the query 104 to the query parser derivation computing device 102; i.e. providing the query reads on forwarding the input message);
determining, at the engine, that the input message includes an unstructured query corresponding to one of the specific contextualization objects based on the general context shift and the unstructured query (Feng, [Fig. 6] note 154, [0047] note FIG. 6 there is shown in block diagram format an example of a query parser derivation process 150 performed by the query parser derivation computing device 102 of FIG. 5… process 150 may receive a query in block 152 and determine if the query is structured or unstructured in block 154);
the number of tokens in the determining being less than if the determining is based on a non-hierarchical contextualization object (Feng, [0020] note The learned model can use class-level multi-gram language model-based features. Such features of multi-gram language models, harvested from tokens contained in a structured queries log, can insulate the model from surface-level tokens, [0064] note Parsing the input query may recognize both a geographic location token and a search term token. Phrase level grammar may be built using a context free grammar composition);
preparing, at the engine, a draft response message including a structured query to at least one of a plurality of search engines corresponding to the one of the specific contextualization objects (Feng, [Fig. 5], [Fig. 6], [0047] note If the query is unstructured the query may be delivered to the system shown in FIG. 5, including the query parser derivation computing device 102 to produce a query parser and parse the query to a structured query);
forwarding the draft response message from the engine to the collaboration platform; sending the structured query from the collaboration platform to a management engine for routing and processing by the at least one of a plurality of search engines (Feng, [Fig. 5], [Fig. 6] note 160, [0047] note parse the query to a structured query, which can then be provided to the search engine in the step of block 160);
receiving, at the collaboration platform, a response to the structured query, from the search engine via the management engine; and, generating an output message responsive to the input message including the draft response message that substitutes the response for the structured query (Feng, [0046] note the search engine may then used the parsed query to perform a search, e.g., in databases 120 to provide a query result 114, e.g., back through the internet 106 to the originator of the query 104; i.e. query results read on an output message).
Feng does not explicitly teach a large language model (LLM).
However, Dong teaches this (Dong, [0024] note a machine learning model(s)—such as a large language model (LLM)—may include a base model(s) and domain specific parts… The base model(s) may include layers that are trained using training data (general-purpose training data) associated with multiple domains. As described herein, a domain may include, but is not limited to, a financing domain, a travel domain, a communications domain, a computer science domain, an automotive domain, an electronics domain, a real estate domain, and/or any other type of domain, [0031] note a part may be trained to understand input data associated with different intents (e.g., booking travel, requesting information, interpreting information, etc.), different tasks (e.g., with regard to booking travel, booking a plane flight, booking a cruise, booking a hotel, etc.), and/or the like).
It would have been obvious to one of ordinary skill in the art at the effective filing date of the application to combine the multi-gram language models of Feng with the LLM including base models and domain specific parts of Dong according to known methods (i.e. using a large language model including base models and domain specific parts). Motivation for doing so is that by activating and/or deactivating domain specific parts of the machine learning model(s) based on the domains for which the machine learning model(s) is deployed, the machine learning model(s) of the current systems may be more accurate, require fewer computing resources, and/or have less latency when processing input data (Dong, [0006]).
Claim 15: Feng and Dong teach the collaboration platform of claim 14 wherein the management engine is incorporated into the collaboration platform (Feng, [0058] note the above disclosed subject matter may be useful for providing local search applications/services (including business, product and utility searches), e.g., with a focus on social media analysis or real-time searches).
Claim 16: Feng and Dong teach the collaboration platform of claim 14 wherein the LLM engine is incorporated into the collaboration platform (Feng, [0058] note the above disclosed subject matter may be useful for providing local search applications/services (including business, product and utility searches), e.g., with a focus on social media analysis or real-time searches).
Claim 17: Feng and Dong teach the collaboration platform of claim 14 wherein the NLP engine is incorporated into the collaboration platform (Feng, [0058] note the above disclosed subject matter may be useful for providing local search applications/services (including business, product and utility searches), e.g., with a focus on social media analysis or real-time searches).
Claim 18: Feng and Dong teach the collaboration platform of claim 14 wherein the NLP engine and LLM engine are combined into a single engine (Dong, [0095] note As shown in FIG. 10, the data center infrastructure layer 1010 may include a resource orchestrator 1012, grouped computing resources 1014, and node computing resources).
Claim 2 is rejected under 35 U.S.C. 103 as being unpatentable over Feng and Dong in further view of Gupta et al., US 2018/0007448 A1 (hereinafter “Gupta”).
Claim 2: Feng and Dong do not explicitly teach the method of claim 1 wherein the general contextualization object configures the LLM engine to classify the unstructured query into at least one of a materials query, a labour query, a delivery query and a general query (Dong, [0024] note a machine learning model(s)—such as a large language model (LLM)—may include a base model(s) and domain specific parts… The base model(s) may include layers that are trained using training data (general-purpose training data) associated with multiple domains. As described herein, a domain may include, but is not limited to… a real estate domain, and/or any other type of domain. A domain specific part may include additional layers that are trained using training data that is specific to the domain)
Feng and Dong do not explicitly teach is based on home renovation planning.
However, Gupta teaches this (Gupta, [0031] note language model database 103 may be configured to store a large number of language models, [0033] note each of the domain specific language models 103a-103d may correspond to specific language used with respect to different genres of content, such as… home renovation).
It would have been obvious to one of ordinary skill in the art at the effective filing date of the application to combine the large language models including domain specific parts of Feng and Dong with the domain specific language models of Gupta according to known methods (i.e. including a domain related specific to home renovation). Motivation for doing so is that this improves existing systems for determining related content based on domain specific language models for more effective identification of said related content (Gupta, [0008]).
Claim 4 is rejected under 35 U.S.C. 103 as being unpatentable over Feng and Dong in further view of Lester et al., US 2023/0325725 A1 (hereinafter “Lester”).
Claim 4: Feng and Dong do not explicitly teach the method of claim 3 wherein a subsequent unstructured query builds on a previous response; and wherein a different specific contextualization object is chosen for the subsequent query than for the original unstructured query.
However, Lester teaches this (Lester, [0028] note inputting the learned prompt and a set of input text into a large pre-trained language model in order to obtain an output that accurately reflects the desired task of the user, [0111] note The input data 502 can include text data, and the pre-trained machine-learned model 510 may be a large natural language processing model (e.g., a T5 model or a GPT-3 model), [0208] note The systems and methods can be repeated iteratively to continue to refine, or tune, the prompt. The user may repeat this process, using the resulting prompt, until the user reaches a result they are happy with).
It would have been obvious to one of ordinary skill in the art at the effective filing date of the application to combine the queries of Feng and Dong with the iterative input refinement of Lester according to known methods (i.e. iteratively refining a user’s query). Motivation for doing so is that the user may repeat this process until the user reaches a result they are happy with (Lester, [0208]).
Claims 6, 7, 9 and 13 are rejected under 35 U.S.C. 103 as being unpatentable over Feng and Dong in further view of QIN, US 2024/0346256 A1 (hereinafter “Qin”).
Claim 6: Feng and Dong do not explicitly teach the method of claim 5 wherein the travel query includes a transportation-actor component that is restricted by an employer policy component.
However, Qin teaches this (Qin, [0016] note LLMs, [0028] note a search engine or chatbot on an internal corporate website may provide employees with accurate responses when presented with a query that is directed to internal or proprietary information. In another example, an external-facing company webpage may provide customers with responses that are focused on the company's products or services, [0039] note Pre-processor 202 may receive contextual information 215 and/or query 216. In embodiments, contextual information 215 may describe the context of the user (e.g., user identifier, user role, user profile, user location, browsing history, etc.) and/or the context of the query (e.g., the current webpage, the product or service associated with the current webpage, query timestamp information, etc.)).
It would have been obvious to one of ordinary skill in the art at the effective filing date of the application to combine the large language models including domain specific parts of Feng and Dong with the contextual information based LLM of Qin according to known methods (i.e. providing an LLM based on contextual information including user profile information, a current webpage and whether the user is an employee associated with a corporate website). Motivation for doing so is that this improves the scope and accuracy of responses generated by an LLM (Qin, [0018]).
Claim 7: Feng, Dong and Qin teach the method of claim 6 wherein the employer policy component corresponds to an employer policy search engine that maintains restrictions as to types of queries to the transportation-actor search engines and the hospitality-actor search engines; the restrictions based on an account from which the input message originates (Qin, [0016] note LLMs, [0028] note a search engine or chatbot on an internal corporate website may provide employees with accurate responses when presented with a query that is directed to internal or proprietary information. In another example, an external-facing company webpage may provide customers with responses that are focused on the company's products or services, [0039] note Pre-processor 202 may receive contextual information 215 and/or query 216. In embodiments, contextual information 215 may describe the context of the user (e.g., user identifier, user role, user profile, user location, browsing history, etc.) and/or the context of the query (e.g., the current webpage, the product or service associated with the current webpage, query timestamp information, etc.)).
Claim 13: Feng and Dong do not explicitly teach the method of claim 11 wherein the travel query includes an account profile of the user generating the input message.
However, Qin teaches this (Qin, [0016] note LLMs, [0028] note a search engine or chatbot on an internal corporate website may provide employees with accurate responses when presented with a query that is directed to internal or proprietary information. In another example, an external-facing company webpage may provide customers with responses that are focused on the company's products or services, [0039] note Pre-processor 202 may receive contextual information 215 and/or query 216. In embodiments, contextual information 215 may describe the context of the user (e.g., user identifier, user role, user profile, user location, browsing history, etc.) and/or the context of the query (e.g., the current webpage, the product or service associated with the current webpage, query timestamp information, etc.)).
It would have been obvious to one of ordinary skill in the art at the effective filing date of the application to combine the large language models including domain specific parts of Feng and Dong with the contextual information based LLM of Qin according to known methods (i.e. providing an LLM based on contextual information including user profile information, a current webpage and whether the user is an employee associated with a corporate website). Motivation for doing so is that this improves the scope and accuracy of responses generated by an LLM (Qin, [0018]).
Claim 9: Feng, Dong and Qin teach the method of claim 7 wherein the coordination is based on aligning a flight schedule with an availability of a ground-transportation service and accommodation (Dong, [0001] note Language models are used in many different applications, such as to schedule travel plans (e.g., booking arrangements for transportation and accommodations etc.), plan activities (e.g., making reservations, etc.), communicate with others (e.g., make phone calls, start video conferences, etc.), shop for items (e.g., purchase items from online marketplaces, etc.), and/or other use cases).
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Giuseppi Giuliani whose telephone number is (571)270-7128. The examiner can normally be reached Monday-Friday.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Aleksandr Kerzhner can be reached at (571)270-1760. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/GIUSEPPI GIULIANI/Primary Examiner, Art Unit 2165
Prosecution Insights