DETAIL OFFICE ACTIONS
The United States Patent & Trademark Office appreciates the response filed for the current application that is submitted on 01/08/2026. The United States Patent & Trademark Office reviewed the following documents submitted and has made the following comments below.
Amendment
Applicant submitted amendments on 12/08/2025. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Information Disclosure Statement
The IDS dated 06/27/2023 has been previously considered and placed in the application file.
Overview
Claims 1, 5-7, 9, and 13-15 are pending in this application and have been considered below.
Claims 16, 20, and 21 have been cancelled.
Claims 1, 5-7, 9, and 13-15 are rejected.
Applicant Arguments:
In regards to the argument on Argument 1, Applicant/s state/s “Claim 21 has been cancelled without prejudice to its further prosecution. Thus, the objection to claim 21 is moot. Withdrawal of the objection is respectfully requested.” therefore, the objection should be withdrawn (See Remarks, page 7, paragraph 2).
In regards to the argument on Argument 2, Applicant/s state/s “Allen et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on metadata associated with the first set of questions and metadata associated with the second set of questions. Therefore, Allen et al. do not teach or suggest machine-readable instructions to cause a machine to select one of a first set of question templates or a second set of question templates based on (a) first metadata associated with the first set of question templates, (b) second metadata associated with the second set of question templates, and (c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1.” therefore, the rejection of 35 U.S.C. 103 should be withdrawn (See Remarks, page 8, paragraph 1).
In regards to the argument on Argument 3, Applicant/s state/s “Adato et al. do not teach or suggest machine readable instructions to cause a machine to select one of a first set of question templates or a second set of question templates based on (a) first metadata associated with the first set of question templates, (b) second metadata associated with the second set of question templates, and ( c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1.therefore, the rejection of 35 U.S.C. 103 should be withdrawn (See Remarks, page 9, paragraph 1).
In regards to the argument on Argument 4, Applicant/s state/s “Allen et al./ Adato et al. combination fails to establish a prima facie case of obviousness against claim 1.” therefore, the rejection of 35 U.S.C. 103 should be withdrawn (See Remarks, page 9, paragraph 2).
In regards to the argument on Argument 5, Applicant/s state/s “The alleged Allen
et al./Adato et al. combination does not teach or suggest such processor circuitry.” therefore, the rejection of 35 U.S.C. 103 should be withdrawn (See Remarks, page 9, paragraph 2).
Examiner’s Responses:
In response to Argument 1, Applicant’s arguments, see Remarks, filed 12/08/25, with respect to the objection of Claim 21 have been fully considered and are persuasive. Therefore, the objection has been withdrawn.
In response to Argument 2, Applicant’s arguments, see Remarks, filed 12/08/25, with respect to the rejection(s) of claim 1 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn due to the amendment. However, upon further consideration, a new ground(s) of rejection is made for Claim 1 and its dependent claims under 35 U.S.C. 103 as obvious over Allen et al (US Patent Pub No. 2018/0157990 A1, hereafter referred to as Allen et al.) in view of Adato et al (U.S Patent Pub. No US 2019/0149725 A1, hereafter referred to as Adato) in further view of Wang et al (U.S Patent Pub. No US 2021/0157854 A, hereafter referred to as Wang).
The Examiner finds that Allen teaches on the amended claim language “based on (a) the first metadata associated with the first set of questions” in Claim 1 with the amendment changing the scope of the claim with the limitation “first metadata associated with a first set of questions”.
Specifically, Allen teaches associating questions with metadata in form of a table in ¶0094-¶0095, Figure 5, 506 which the examiner interprets as meta data associated with a question. Applicant argues that “Allen et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on metadata associated with the first set of questions and metadata associated with the second set of questions. Therefore, Allen et al. do not teach or suggest machine-readable instructions to cause a machine to select one of a first set of question templates or a second set of question templates based on (a) first metadata associated with the first set of question templates, (b) second metadata associated with the second set of question templates, and (c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1”. However, the Examiner interprets that Allen teaches the main concept of using non-transitory machine- readable medium comprising machine-readable instructions to conduct image analysis using a question answer trained machine learning model, the additional details of the function and characteristics of the main concepts as stated above by the applicant in the amendments is taught by Wang in the details of the rejection below. The Examiner will maintain prior art Allen and details of the rejection are below.
In response to Argument 3, Applicant’s arguments, see Remarks, filed 12/08/25, with respect to the rejection(s) of claim 1 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn due to the amendment. However, upon further consideration, a new ground(s) of rejection is made for Claim 1 and its dependent claims under 35 U.S.C. 103 as obvious over Allen et al (US Patent Pub No. 2018/0157990 A1, hereafter referred to as Allen et al.) in view of Adato et al (U.S Patent Pub. No US 2019/0149725 A1, hereafter referred to as Adato) in further view of Wang et al (U.S Patent Pub. No US 2021/0157854 A, hereafter referred to as Wang).
The Examiner finds that Adato teaches on the amended claim language “(c) the image type” in Claim 1 with the amendment changing the scope of the claim with the limitation “with a respective one of the first metadata or the second metadata”.
Specifically, Adato teaches images and videos being taken of a shelving unit or the entire store in ¶0823, ¶0395, Fig 23 and Fig 6c. Applicant argues that “Adato et al. do not teach or suggest machine readable instructions to cause a machine to select one of a first set of question templates or a second set of question templates based on (a) first metadata associated with the first set of question templates, (b) second metadata associated with the second set of question templates, and ( c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1.therefore, the rejection of 35 U.S.C. 103 should be withdrawn”. However, the Examiner interprets that Adato teaches the concept of using different types of images specifically in a retail store format in regards to images of shelves, the additional details of the function and characteristics of the main concepts as stated above by the applicant in the amendments is taught by Allen and Wang in the details of the rejection below. The Examiner will maintain prior art Adato and details of the rejection are below.
In response to Argument 4, Applicant’s arguments, see Remarks, filed 12/08/25, with respect to the rejection(s) of claim 1 under 35 U.S.C. 103 have been fully considered and are persuasive. Therefore, the rejection has been withdrawn due to the amendment. However, upon further consideration, a new ground(s) of rejection is made for Claim 1 and its dependent claims under 35 U.S.C. 103 as obvious over Allen et al (US Patent Pub No. 2018/0157990 A1, hereafter referred to as Allen et al.) in view of Adato et al (U.S Patent Pub. No US 2019/0149725 A1, hereafter referred to as Adato) in further view of Wang et al (U.S Patent Pub. No US 2021/0157854 A, hereafter referred to as Wang).
The Examiner finds that Allen teaches on the amended claim language “based on (a) the first metadata associated with the first set of questions” in Claim 1 with the amendment changing the scope of the claim with the limitation “first metadata associated with a first set of questions”. Adato teaches on the amended claim language “(c) the image type” in Claim 1 with the amendment changing the scope of the claim with the limitation “with a respective one of the first metadata or the second metadata”.
Specifically, Allen teaches associating questions with metadata in form of a table in ¶0094-¶0095, Figure 5, 506 which the examiner interprets as meta data associated with a question. Applicant argues that “Allen et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on metadata associated with the first set of questions and metadata associated with the second set of questions. Therefore, Allen et al. do not teach or suggest machine-readable instructions to cause a machine to select one of a first set of question templates or a second set of question templates based on (a) first metadata associated with the first set of question templates, (b) second metadata associated with the second set of question templates, and (c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1”. However, the Examiner interprets that Allen teaches the main concept of using non-transitory machine- readable medium comprising machine-readable instructions to conduct image analysis using a question answer trained machine learning model, the additional details of the function and characteristics of the main concepts as stated above by the applicant in the amendments is taught by Wang in the details of the rejection below.
Specifically, Adato teaches images and videos being taken of a shelving unit or the entire store in ¶0823, ¶0395, Fig 23 and Fig 6c. Applicant argues that “Adato et al. do not teach or suggest machine readable instructions to cause a machine to select one of a first set of question templates or a second set of question templates based on (a) first metadata associated with the first set of question templates, (b) second metadata associated with the second set of question templates, and ( c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1.therefore, the rejection of 35 U.S.C. 103 should be withdrawn”. However, the Examiner interprets that Adato teaches the concept of using different types of images specifically in a retail store format in regards to images of shelves, the additional details of the function and characteristics of the main concepts as stated above by the applicant in the amendments is taught by Allen and Wang in the details of the rejection below. Examiner has met all requirements establishing a prima facie case: all factual findings required by Graham were supplied in the previous and present Actions; the references are related art, and Applicant has supplied no evidence that there is no reasonable expectation of success; all claim limitations were met in the previous and present Actions, and Applicant has merely made the allegation that the limitations are not met, and thus has not provided any evidence or argument directed to how the identified elements in the first action fail to meet the claimed limitations or to how the identified elements are otherwise distinguishable from the claimed limitations. Neither has Applicant supplied any evidence or argument addressing any failure of Examiner's application of the TSM test, pursuant to current governing law (see KSR International Co. v. Teleflex Inc., 82 USPQ2d 1385 (U.S. 2007)). The Examiner will maintain prior art Allen and Adato and details of the rejection are below.
In response to Argument 5, Applicant’s arguments, see Remarks, filed 12/08/25, with respect to the rejection(s) of claim 9 under 35 U.S.C. 103 have been considered but are moot in view of new ground(s) of rejection caused by the amendments. Upon further consideration, a new ground(s) of rejection is made for Claim 9 and its dependent claims under 35 U.S.C. 103 as obvious over Allen et al (US Patent Pub No. 2018/0157990 A1, hereafter referred to as Allen et al.) in view of Adato et al (U.S Patent Pub. No US 2019/0149725 A1, hereafter referred to as Adato) in further view of Wang et al (U.S Patent Pub. No US 2021/0157854 A, hereafter referred to as Wang).
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness
rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5-7, 9, 13-15 are rejected under 35 U.S.C. 103 as obvious over Allen et al (US Patent Pub No. 2018/0157990 A1, hereafter referred to as Allen et al.) in view of Adato et al (U.S Patent Pub. No US 2019/0149725 A1, hereafter referred to as Adato) in further view of Wang et al (U.S Patent Pub. No US 2021/0157854 A, hereafter referred to as Wang).
Regarding Claim 1, Allen et al. teaches at least one non-transitory machine- readable medium comprising machine-readable instructions (¶0015, Allen et al. "Computer readable program instructions for carrying out operations of the present invention"), to cause a machine to at least:
with respective store dictionaries (¶0094, Allen et al "followed by the receipt of a corpus of human-readable text, such as a collection of documents, in step 504." Figure 5a, 504 "Receive Corpus Of Human-Readable Text");
of questions (¶0095, Allen et al "associated with each of the parsed column and row labels, individually or in combination, is identified in step 518 and then assigned in step 520 as metadata to each cell within the table." Figure 5b, 520 "Assign Identified Content Types As Metadata ,Associated With Various Cells In The Table");
based on (a) the first metadata associated with the first set of questions, (¶0095, Allen et al "associated with each of the parsed column and row labels, individually or in combination, is identified in step 518 and then assigned in step 520 as metadata to each cell within the table." Figure 5b, 520 "Assign Identified Content Types As Metadata ,Associated With Various Cells In The Table")
to generate query responses (¶0034, Allen at al. "The QA system 100 then generates an output response or answer 120 with the final answer and associated confidence and supporting evidence.").
Allen et al. do not explicitly teach categorize store images based on an image type;
(c )the image type the image type being one of a whole image type associated with two or more shelves or a cropped image type.
However, Adato is in the same field of image processing used for retail store. Further, Adato teaches categorize store images (Adato ¶0283 discloses categorizing at least one image using classification algorithms) based on an image type (Adato ¶0823 discloses images or video being taken of a shelving unit or the whole store);
and (c )the image type (Adato ¶0823 discloses images or video being taken of a shelving unit or the whole store), the image type being one of a whole image type associated with two or more shelves (Adato ¶0395 discloses receiving an image of an entire shelf, Fig 23 discloses an image of two or more shelves, Fig 6c discloses an images of two or more shelves), or a cropped image type (Adato ¶0152, ¶0736 discloses the input image being cropped Fig 46A shows the second image being a cropped image of a single shelf).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen to include categorizing store images based on the type of image and the entire or cropped image pertaining the number of shelves in the image as taught by Adato, to make the invention a method that could use the question answer machine learning model that was created to answer questions about the store observation images based on the number of shelves in the image; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that could be used on store observation images since there is a need for an continuous monitoring of dynamically changing product displays, and to increase productivity, and to provide a dynamic solution that will automatically monitor retail spaces. (Adato, ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Allen and Adato in combination do not explicitly disclose obtain metadata associated, select ones of the store dictionaries for use based on the respective metadata select ones of the store dictionaries for use based on the respective metadata obtain first metadata associated with a first set of questions and second metadata associated with a second set train a machine-learning model based on question-answer pairs the question-answer pairs generated by;
selecting one of the first set of questions or the second set of questions and (b) the second metadata associated with the second set of questions associated with a respective one of the first metadata or the second metadata,
generating question-answer pairs based on the selected ones of the set of the store dictionaries and the selected one of the first set of questions or the second set of questions
and
execute the trained machine-learning model.
Wang is in the same field of processing search queries. Further, Wang teaches obtain metadata associated (Wang Fig 8, ¶0086, and discloses extracting metadata from the acquired text documents which the examiner interprets as equivalent to dictionaries)
select ones of the store dictionaries (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) for use based on the respective metadata (Wang Fig 8, ¶0086, and discloses extracting metadata from the acquired text documents which the examiner interprets as equivalent to dictionaries)
obtain first metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with a first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) and second metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with a second set (Wang ¶0020 and Fig 18 disclose a second set of questions)
train a machine-learning model based on question-answer pairs (Wang Fig 16 discloses training a machine learning model based off question answer pairs), the question-answer pairs generated by (Wang Fig 19 discloses the generation of question and answer pairs):
selecting one (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
and (b) the second metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
associated with a respective one of the first metadata (Wang ¶0047-¶0048 disclose fields associated with each query) or the second metadata (Wang ¶0047-¶0048 disclose fields associated with each query),
generating question-answer pairs (Wang Fig 19 discloses the generation of question and answer pairs)based on the selected ones of the set of the store dictionaries (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) and the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions)or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
execute the trained machine-learning model (Wang ¶0073 disclose performing a search query using the ML model).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen in view of Adato to include the first and second set of questions associated with respective metadata as taught by Wang, to make the invention a method that could use the question answer machine learning model trained on question answer pairs to answer questions about the store observation images; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that could be used on store observation images since there is a need to enables users to intuitively search unstructured data using natural language, that returns specific and personalized answers to questions, giving end users an experience that comes closer to interacting with a human expert. (Wang, ¶0026).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 5, Allen in view of Adato in further view of Wang teaches the at least one non-transitory machine-readable medium as defined in claim 1, wherein the machine-readable instructions, are to cause the machine to mark the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions) for association (¶0093, Allen et al. "A lower initial ranking score (e.g., score=0.5) is then assigned to each QA pair generated using a template that is based on a question referenced to or created from the extracted portion of unstructured text.") with ones of the categorized store images (Adato ¶0283 discloses categorizing at least one image taken from the store using classification algorithms). See rationale for Claim 1, its parent claim.
Regarding Claim 6, Allen in view of Adato in further view of Wang teaches the at least one non-transitory machine-readable medium as defined in claim 1, wherein the machine-readable instructions, are to cause the machine to generate the question-answer pairs (Wang Fig 19 discloses the generation of question and answer pairs) by determining natural language variations (¶0021, Allen et al. "The Named Entity subsystem 112 receives and processes each question 111 by using natural language processing (NLP) to analyze each question and extract question topic information contained in the question") of the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions). See rationale for Claim 1, its parent claim.
Regarding Claim 7, Allen in view of Adato in further view of Wang teaches the at least one non-transitory machine-readable medium as defined in claim 6, wherein the
the selected one (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions) is included in a store observation dataset (¶0024 Allen et al. discloses big datasets coming from a variety of sources) is associated with the store dictionaries (¶0094, Allen et al "followed by the receipt of a corpus of human-readable text, such as a collection of documents, in step 504." Figure 5a, 504 "Receive Corpus Of Human-Readable Text"). See rationale for Claim 1, its parent claim.
Regarding Claim 9, Allen et al. teaches an apparatus to generate query responses comprising (¶0037, Allen et al. "An illustrative example of an information processing system showing an exemplary processor and various components commonly accessed by the processor is shown in Fig. 2"):
interface circuitry (¶0027, Allen et al. discloses "likewise include potentially any other computing device elements generally known in the art including buses, storage devices, communication interfaces, and the like");
machine readable instructions (¶0015, Allen et al. "Computer readable program instructions for carrying out operations of the present invention"); and
to be programmed by the machine readable instructions (¶0012, Allen et al. "Furthermore, aspects of the present invention may take the form of computer program product embodied in a computer readable storage medium, or media, having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.") to:
with respective store dictionaries (¶0094, Allen et al "followed by the receipt of a corpus of human-readable text, such as a collection of documents, in step 504." Figure 5a, 504 "Receive Corpus Of Human-Readable Text");
of questions (¶0095, Allen et al "associated with each of the parsed column and row labels, individually or in combination, is identified in step 518 and then assigned in step 520 as metadata to each cell within the table." Figure 5b, 520 "Assign Identified Content Types As Metadata ,Associated With Various Cells In The Table");
based on (a) the first metadata associated with the first set of questions, (¶0095, Allen et al "associated with each of the parsed column and row labels, individually or in combination, is identified in step 518 and then assigned in step 520 as metadata to each cell within the table." Figure 5b, 520 "Assign Identified Content Types As Metadata ,Associated With Various Cells In The Table")
based on the question-answer pairs (¶0102, Allen et al. "The ranked QA pairs are then used in step 572 to train a QA system" ¶0049, Allen et al. "As used herein, ground truth broadly refers to a set of question-answer (QA) pairs used to train a machine learning system, such as a QA system, where each question of an associated QA pair has a corresponding correct answer."); and
generate query (¶0034, Allen at al. "The QA system 100 then generates an output response or answer 120 with the final answer and associated confidence and supporting evidence.").
Allen et al. do not explicitly teach categorize store images based on an image type the image type being one of a whole image type associated with two or more shelves, or a cropped image type (c) the image type.
However, Adato is in the same field of image processing used for retail store. Further, Adato teaches categorize store images (Adato ¶0283 discloses categorizing at least one image using classification algorithms) based on an image type (Adato ¶0823 discloses images or video being taken of a shelving unit or the whole store) the image type being one of a whole image type associated with two or more shelves (Adato ¶0395 discloses receiving an image of an entire shelf, Fig 23 discloses an image of two or more shelves, Fig 6c discloses an images of two or more shelves), or a cropped image type (Adato ¶0152, ¶0736 discloses the input image being cropped Fig 46A shows the second image being a cropped image of a single shelf) (c )the image type (Adato ¶0823 discloses images or video being taken of a shelving unit or the whole store).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen to include categorizing store images based on the type of image and the entire or cropped image pertaining the number of shelves in the image as taught by Adato, to make the invention a method that could use the question answer machine learning model that was created to answer questions about the store observation images based on the number of shelves in the image; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that could be used on store observation images since there is a need for an continuous monitoring of dynamically changing product displays, and to increase productivity, and to provide a dynamic solution that will automatically monitor retail spaces. (Adato, ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Allen and Adato in combination do not explicitly disclose at least one processor circuit,
obtain metadata associated for use based on the respective metadata,
obtain first metadata associated with a first set of questions and second metadata associated with a second set;
select one of the first set of questions or the second set of questions and (b)the second metadata with the second set of questions associated with a respective one of the first metadata or the second metadata,
generate question-answer pairs based on the selected ones of the set of the store dictionaries and the selected one of the first set of questions or the second set of questions
train a machine-learning model based on using the trained machine-learning model
Wang is in the same field of processing search queries. Further, Wang teaches at least one processor circuit (Wang ¶0136-¶0137 discloses the use of processor in hardware such as RAM which a person of ordinary skill in the art would understand to have circuitry)
obtain metadata associated (Wang Fig 8, ¶0086, and discloses extracting metadata from the acquired text documents which the examiner interprets as equivalent to dictionaries)
for use based on the respective metadata (Wang Fig 8, ¶0086, and discloses extracting metadata from the acquired text documents which the examiner interprets as equivalent to dictionaries)
obtain first metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with a first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) and second metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with a second set (Wang ¶0020 and Fig 18 disclose a second set of questions)
select one (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
and (b)the second metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
associated with a respective one of the first metadata (Wang ¶0047-¶0048 disclose fields associated with each query) or the second metadata (Wang ¶0047-¶0048 disclose fields associated with each query),
generate question-answer pairs (Wang Fig 19 discloses the generation of question and answer pairs)based on the selected ones of the set of the store dictionaries (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) and the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions)or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
train a machine-learning model (Wang Fig 16 discloses training a machine learning model based off question answer pairs),
based on using the trained machine-learning model (Wang ¶0073 disclose performing a search query using the ML model).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen in view of Adato to include the first and second set of questions associated with respective metadata as taught by Wang, to make the invention a method that could use the question answer machine learning model trained on question answer pairs to answer questions about the store observation images; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that could be used on store observation images since there is a need to enables users to intuitively search unstructured data using natural language, that returns specific and personalized answers to questions, giving end users an experience that comes closer to interacting with a human expert. (Wang, ¶0026).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 13, Allen in view of Adato in further view of Wang teaches the apparatus as defined in claim 9, wherein one or more of the at least one processor circuit (¶0015, Allen et al. " electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program") is to identify
the image type (Adato ¶0783 discloses the system identifying image data) as the whole image type (Adato ¶0395 discloses receiving an image of an entire shelf) or the cropped image type, the cropped image type associated with a single retail shelf(Adato ¶0152, ¶0736 discloses the input image being cropped Fig 46A shows the second image being a cropped image of a single shelf) .See rationale for Claim 1, its parent claim.
Regarding Claim 14, Allen in view of Adato in further view of Wang teaches the apparatus as defined in claim 9, wherein one or more of the at least one processor circuit (¶0015, Allen et al. " electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program") is to mark the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions) (¶0093, Allen et al. "A lower initial ranking score (e.g., score=0.5) is then assigned to each QA pair generated using a template that is based on a question referenced to or created from the extracted portion of unstructured text.") with ones of the store images (Adato ¶0283 discloses categorizing at least one image taken from the store using classification algorithms).See rationale for Claim 1, its parent claim.
Regarding Claim 15, Allen in view of Adato in further view of Wang teaches the apparatus as defined in claim 9, wherein one or more of the at least one processor circuit (¶0015, Allen et al. " electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program") is to generate the question-answer pairs by determining natural language variations of the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions) (¶0021, Allen et al. "The Named Entity subsystem 112 receives and processes each question 111 by using natural language processing (NLP) to analyze each question and extract question topic information contained in the question"). See rationale for Claim 1, its parent claim.
Claim 8 is rejected under 35 U.S.C. 103 as obvious over Allen in view of Adato in view of Wang in further view of Wang et al (US Patent Pub US 2018/0314689 A1 hereafter referred to as Wang 2).
Regarding Claim 8, Allen in view of Adato in view of Wang the at least one non-transitory machine-readable medium (¶0015, Allen et al. "Computer readable program instructions for carrying out operations of the present invention") as defined in claim 1, obtained from a store observation dataset (Adato ¶0142, ¶discloses a database that contains information about the product placement on the store shelves and other related store observation data).
Allen et al. in view of Adato in view of Wang do not explicitly teach wherein the machine-readable instructions are to cause the machine to train the machine-learning model using an updated dictionary.
However, Wang 2 is in the same field of image processing. Further, Wang 2 teaches wherein the machine-readable instructions are to cause the machine to train the machine-learning model using an updated dictionary (¶0080, Wang et al. "In these implementations, the models, grammars, dictionaries, ontologies, etc. from a well-developed system can be translated, using machine translation, into the multiple languages that are to be supported by a device. The translated models, grammars, etc. can be refined by additional training and/or machine learning techniques.").
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen in view of Adato in view of Wang to include a store dictionary that is updated as taught by Wang 2, to make the invention a method that could use the question answer machine learning model that was created to expand its range using the store dictionary; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that can utilize the store dictionaries since there is a need for increasing the confidence value for the translated text when the additional verbal input affirms the translated text. (Wang 2, ¶0015).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to RACHEL LYNN ROBERTS whose telephone number is (571)272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL L ROBERTS/
Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674