DETAILED ACTION
The United States Patent & Trademark Office appreciates the response filed for the current application that is submitted on 05/01/2026. The United States Patent & Trademark Office reviewed the following documents submitted and has made the following comments below.
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Overview
The United States Patent & Trademark Office appreciates the response filed for the current application that is submitted on 05/01/2026. The United States Patent & Trademark Office reviewed the following documents submitted and has made the following comments below.
Claims are 1, 5-9, and 13-15 are pending in this application and have been considered below.
Claims are 1, 5-9, and 13-15 are rejected.
Claims 2-4, 10-12, and 16-21 have been cancelled.
Amendment
Applicant submitted amendments on 05/01/2026. The Examiner acknowledges the amendment and has reviewed the claims accordingly.
Information Disclosure Statement
The IDS dated 06/27/2023 has been previously considered and placed in the application file.
Applicant Arguments:
In regards to Argument 1, Applicant/s state/s that “However, Allen et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on the type of content associated with the parsed columns and rows. Therefore, Allen et al. do not teach or suggest machine-readable instructions to select one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1.” Therefore the 35 U.S.C 103 rejection should be withdrawn with respect to Claim 1 (See Remarks Pg 7, ¶01).
In regards to Argument 2, Applicant states that “However, Adato et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on the image analysis. Therefore, Allen et al. do not teach or suggest machine-readable instructions to select one of a first set of questions or a second set of questions based on ( a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and (c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1. Accordingly, the Allen et al./ Adato et al. combination does not teach or suggest the machine-readable instructions of claim 1.” Therefore the 35 U.S.C 103 rejection should be withdrawn with respect to Claim 1 (See Remarks Pg 7, ¶02).
In regards to Argument 3, Applicant states that “However, suggesting a subset of user queries, document candidates, and/or answer candidates based on confidence scores or diversity sampling as mentioned by Wang et al. does not teach or suggest selecting one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata. Moreover, Wang et al. do not teach or suggest selecting one of the questions in FIG. 17 or the questions in FIG. 18, much less selecting one of the questions of FIGS. 17 or 18 based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata.” Therefore the 35 U.S.C 103 rejection should be withdrawn with respect to Claim 1 (See Remarks Pg 8, ¶02).
In regards to Argument 4, Applicant states that “Wang et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on the fields. Therefore, Wang et al. do not teach or suggest machine-readable instructions to cause a machine to select one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1. Accordingly, Wang et al. do not cure the deficiencies of the Allen et al./ Adato et al. combination.” Therefore the 35 U.S.C 103 rejection should be withdrawn with respect to Claim 1 (See Remarks Pg 9, ¶01).
In regards to Argument 5, Applicant states that “the Office action contains no reasoning as to why one of ordinary skill in the art would be motivated to combine Allen et al., Wang et al., and Adato et al. to obtain machine-readable instructions to cause a machine to select one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1.” Therefore the 35 U.S.C 103 rejection should be withdrawn with respect to Claim 1 (See Remarks Pg 10, ¶01).
In regards to Argument 6, Applicant states that “Independent claim 9 sets forth at least one processor circuit to select one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata. The alleged Allen et al./ Adato et al./W ang et al. combination does not teach or suggest such processor circuitry.” Therefore the 35 U.S.C 103 rejection should be withdrawn with respect to Claim 9 (See Remarks Pg 10, ¶03).
Examiner’s Responses:
In response to Argument 1, Applicant’s arguments, see Remarks, filed 05/01/2026 with respect to claim 1, the Examiner has fully considered the argument and has not found it persuasive. In response, the Examiner respectfully disagrees. The Examiner interprets that Allen in view of Adato in view of Wang teaches on the selecting of metadata described in Claim 1. Specifically, Allen teaches that metadata is associated with columns and rows which form the first set of questions in Fig 5B, 520-524 and ¶0095. Allen also teaches implementing programs with a computer readable medium in ¶0012, ¶0013, ¶0014, ¶0015. Additionally, Wang further details selecting the first or second set of questions in Figs 15-18 and ¶0102, ¶0020. Additionally, Adato teaches the image type consistent with what is specified in the claim as a whole or cropped image in ¶0823, ¶0395 and Fig 23. Applicant argues that “Allen et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on the type of content associated with the parsed columns and rows. Therefore, Allen et al. do not teach or suggest machine-readable instructions to select one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1.” However, we determine claim scope not solely on the basis of claim language, but also on giving claims their broadest reasonable construction in light of the specification as it would be interpreted by one of ordinary skill in the art. In re Am. Acad. of Sci. Tech. Ctr., 367 F.3d 1359, 1364 (Fed. Cir. 2004). See also Superguide Corp. v. DirecTV Enterprises, Inc., 358 F.3d 870, 875 (Fed. Cir. 2004) (“Though understanding the claim language may be aided by explanations contained in the written description, it is important not to import into a claim limitations that are not part of the claim.”). The Examiner interprets that under broadest reasonable interpretation “metadata” has no special definition in the claims, and therefore can be interpreted as any type of data pertaining to the question answer data, including the metadata taught by Allen. Allen specifically teaches metadata being used to form the question answer pairs in Figure 5, which the examiner is interpreting as metadata associated with question answer pairs. Therefore, the Examiner interprets that Allen teaches the main concept of using metadata associated with questions and implementing the method using machine readable instructions, the additional details of the function and characteristics of the main concepts as stated above by the applicant is taught by Adato and Wang. The Examiner will maintain prior arts Allen, Wang, and Adato.
In response to Argument 2, Applicant’s arguments, see Remarks, filed 05/01/2026 with respect to claim 1, the Examiner has fully considered the argument and has not found it persuasive. In response, the Examiner respectfully disagrees. The Examiner interprets that Allen in view of Adato in view of Wang teaches selecting one of a first set of questions or a second set of questions based on the image analysis described in Claim 1. Specifically, Allen teaches that metadata is associated with columns and rows which form the first set of questions in Fig 5B, 520-524 and ¶0095. Allen also teaches implementing programs with a computer readable medium in ¶0012, ¶0013, ¶0014, ¶0015. Additionally, Wang further details selecting the first or second set of questions in Figs 15-18 and ¶0102, ¶0020. Additionally, Adato teaches the image type consistent with what is specified in the claim as a whole or cropped image in ¶0823, ¶0395 and Fig 23. Applicant argues that “Adato et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on the image analysis. Therefore, Allen et al. do not teach or suggest machine-readable instructions to select one of a first set of questions or a second set of questions based on ( a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and (c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1. Accordingly, the Allen et al./ Adato et al. combination does not teach or suggest the machine-readable instructions of claim 1.” In response to applicant's argument that the references fail to show certain features of applicant’s invention, it is noted that the features upon which applicant relies (i.e. image analysis) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Allen specifically teaches metadata being used to form the question answer pairs in Figure 5, which the examiner is interpreting as metadata associated with question answer pairs and implementing the method with machine readable instructions in ¶0012-¶0017. Therefore, the Examiner interprets that Allen teaches the main concept of using metadata associated with questions and implementing the method using machine readable instructions, the additional details of the function and characteristics of the main concepts as stated above by the applicant is taught by Adato and Wang. The Examiner will maintain prior arts Allen, Wang, and Adato.
In response to Argument 3, Applicant’s arguments, see Remarks, filed 05/01/2026 with respect to claim 1, the Examiner has fully considered the argument and has not found it persuasive. In response, the Examiner respectfully disagrees. The Examiner interprets that Allen in view of Adato in view of Wang teaches on the selecting of metadata described in Claim 1. Specifically, Allen teaches that metadata is associated with columns and rows which form the first set of questions in Fig 5B, 520-524 and ¶0095. Allen also teaches implementing programs with a computer readable medium in ¶0012, ¶0013, ¶0014, ¶0015. Additionally, Wang further details selecting the first or second set of questions in Figs 15-18 and ¶0102, ¶0020. Additionally, Adato teaches the image type consistent with what is specified in the claim as a whole or cropped image in ¶0823, ¶0395 and Fig 23. Applicant argues that ““However, suggesting a subset of user queries, document candidates, and/or answer candidates based on confidence scores or diversity sampling as mentioned by Wang et al. does not teach or suggest selecting one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata. Moreover, Wang et al. do not teach or suggest selecting one of the questions in FIG. 17 or the questions in FIG. 18, much less selecting one of the questions of FIGS. 17 or 18 based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata.” However, we determine claim scope not solely on the basis of claim language, but also on giving claims their broadest reasonable construction in light of the specification as it would be interpreted by one of ordinary skill in the art. In re Am. Acad. of Sci. Tech. Ctr., 367 F.3d 1359, 1364 (Fed. Cir. 2004). See also Superguide Corp. v. DirecTV Enterprises, Inc., 358 F.3d 870, 875 (Fed. Cir. 2004) (“Though understanding the claim language may be aided by explanations contained in the written description, it is important not to import into a claim limitations that are not part of the claim.”). The Examiner interprets that under broadest reasonable interpretation “selecting” has no special definition in the claims, and therefore can be interpreted as selecting a proper subset of data as taught Wang in Fig 15 and ¶0102. Wang also teaches a first and second set of questions which under broadest reasonable interpretation can be interpreted as a subset of data. Therefore, the Examiner interprets that Wang teaches the main concept of selecting a set of questions, the additional details of the function and characteristics of the main concepts as stated above by the applicant is taught by Adato and Allen. The Examiner will maintain prior arts Allen, Wang, and Adato.
In response to Argument 4, Applicant’s arguments, see Remarks, filed 05/01/2026 with respect to claim 1, the Examiner has fully considered the argument and has not found it persuasive. In response, the Examiner respectfully disagrees. The Examiner interprets that Allen in view of Adato in view of Wang teaches on selecting one of a first set of questions or a second set of questions based on the fields described in Claim 1. Specifically, Allen teaches that metadata is associated with columns and rows which form the first set of questions in Fig 5B, 520-524 and ¶0095. Allen also teaches implementing programs with a computer readable medium in ¶0012, ¶0013, ¶0014, ¶0015. Additionally, Wang further details selecting the first or second set of questions in Figs 15-18 and ¶0102, ¶0020. Additionally, Adato teaches the image type consistent with what is specified in the claim as a whole or cropped image in ¶0823, ¶0395 and Fig 23. Applicant argues that “Wang et al. do not teach or suggest selecting one of a first set of questions or a second set of questions based on the fields. Therefore, Wang et al. do not teach or suggest machine-readable instructions to cause a machine to select one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata, as set forth in claim 1. Accordingly, Wang et al. do not cure the deficiencies of the Allen et al./ Adato et al. combination.” In response to applicant's argument that the references fail to show certain features of applicant’s invention, it is noted that the features upon which applicant relies (i.e. fields) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). However, we determine claim scope not solely on the basis of claim language, but also on giving claims their broadest reasonable construction in light of the specification as it would be interpreted by one of ordinary skill in the art. In re Am. Acad. of Sci. Tech. Ctr., 367 F.3d 1359, 1364 (Fed. Cir. 2004). See also Superguide Corp. v. DirecTV Enterprises, Inc., 358 F.3d 870, 875 (Fed. Cir. 2004) (“Though understanding the claim language may be aided by explanations contained in the written description, it is important not to import into a claim limitations that are not part of the claim.”). The Examiner interprets that under broadest reasonable interpretation “selecting” has no special definition in the claims, and therefore can be interpreted as selecting a proper subset of data as taught Wang in Fig 15 and ¶0102. Wang also teaches a first and second set of questions which under broadest reasonable interpretation can be interpreted as a subset of data. Therefore, the Examiner interprets that Wang teaches the main concept of selecting a set of questions, the additional details of the function and characteristics of the main concepts as stated above by the applicant is taught by Adato and Allen. The Examiner will maintain prior arts Allen, Wang, and Adato.
In response to Argument 5, Applicant’s arguments, see Remarks, filed 05/01/2026 with respect to claim 1, the Examiner has fully considered the argument and has not found it persuasive. The Examiner respectfully disagrees. The Examiner made a proper determination of obviousness under 35 U.S.C. §103, and also provided an appropriate supporting rationale in view of the decision by the Supreme Court in KSR International Co. v. Teleflex Inc. (KSR), 550 U.S. 398, 82 USPQ2d 1385 (2007). The Examiner’s rational are based on the Office’s current understanding of the law, and are believed to be fully consistent with the binding precedent of the Supreme Court. Furthermore, the Examiner supported the rejection under 35 U.S.C. §103 via making the clear articulation of the reason(s) why the claimed invention would have been obvious by citing the specific areas in the prior art references. Further the Examiner, clearly stating the modification of the inventions, supported the rejection under 35 U.S.C. §103 by making the analysis explicit. Last, the Examiner did not make conclusory statements. The Court quoting In re Kahn, 441 F.3d 977, 988, 78 USPQ2d 1329, 1336 (Fed. Cir. 2006), stated that “‘[R]ejections on obviousness cannot be sustained by mere conclusory statements; instead, there must be some articulated reasoning with some rational underpinning to support the legal conclusion of obviousness.’” KSR, 550 U.S. at, 82 USPQ2d at 1396. Therefore, the Examiner has established a proper 35 U.S.C. §103 rejection with Allen in view of Adato in further view of Wang, which is disclosed in detail below.
In response to Argument 6, Applicant’s arguments, see Remarks, filed 05/01/2026 with respect to claim 9, the Examiner has fully considered the argument and has not found it persuasive. In response, the Examiner respectfully disagrees. The Examiner interprets that Allen in view of Adato in view of Wang teach a processor circuit implementing select one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata. Specifically, Allen teaches that metadata is associated with columns and rows which form the first set of questions in Fig 5B, 520-524 and ¶0095. Allen also teaches a processor as part of a computer implementing programs with a computer readable medium in ¶0012, ¶0013, ¶0014, ¶0015 and ¶0017. Additionally, Wang further details selecting the first or second set of questions in Figs 15-18 and ¶0102, ¶0020 and the use of a processor in hardware to implement a program in ¶0057, ¶0136-¶0137 and Fig 22. Additionally, Adato teaches the image type consistent with what is specified in the claim as a whole or cropped image in ¶0823, ¶0395 and Fig 23. Applicant argues that “Independent claim 9 sets forth at least one processor circuit to select one of a first set of questions or a second set of questions based on (a) first metadata associated with the first set of questions, (b) second metadata associated with the second set of questions, and ( c) an image type associated with a respective one of the first metadata or the second metadata. The alleged Allen et al./ Adato et al./W ang et al. combination does not teach or suggest such processor circuitry.” However, we determine claim scope not solely on the basis of claim language, but also on giving claims their broadest reasonable construction in light of the specification as it would be interpreted by one of ordinary skill in the art. In re Am. Acad. of Sci. Tech. Ctr., 367 F.3d 1359, 1364 (Fed. Cir. 2004). See also Superguide Corp. v. DirecTV Enterprises, Inc., 358 F.3d 870, 875 (Fed. Cir. 2004) (“Though understanding the claim language may be aided by explanations contained in the written description, it is important not to import into a claim limitations that are not part of the claim.”). The Examiner interprets that under broadest reasonable interpretation “processor circuit” has no special definition in the claims, and therefore can be interpreted as any hardware that performs processing which is taught by both Wang and Allen. Therefore, the Examiner interprets that Allen teaches the main concept of selecting a set of questions, the additional details of the function and characteristics of the main concepts as stated above by the applicant is taught by Adato and Wang. The Examiner will maintain prior arts Allen, Wang, and Adato.
The Examiner proposed an Examiner Amendment on 05/27/2026. However, the Examiner’s Amendment was not agreed upon.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness
rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1, 5-7, 9, 13-15 are rejected under 35 U.S.C. 103 as obvious over Allen et al (US Patent Pub No. 2018/0157990 A1, hereafter referred to as Allen et al.) in view of Adato et al (U.S Patent Pub. No US 2019/0149725 A1, hereafter referred to as Adato) in further view of Wang et al (U.S Patent Pub. No US 2021/0157854 A, hereafter referred to as Wang).
Regarding Claim 1, Allen et al. teaches at least one non-transitory machine- readable medium comprising machine-readable instructions (¶0015, Allen et al. "Computer readable program instructions for carrying out operations of the present invention"), to cause a machine to at least:
with respective store dictionaries (¶0094, Allen et al "followed by the receipt of a corpus of human-readable text, such as a collection of documents, in step 504." Figure 5a, 504 "Receive Corpus Of Human-Readable Text");
of questions (¶0095, Allen et al "associated with each of the parsed column and row labels, individually or in combination, is identified in step 518 and then assigned in step 520 as metadata to each cell within the table." Figure 5b, 520 "Assign Identified Content Types As Metadata ,Associated With Various Cells In The Table");
based on (a) the first metadata associated with the first set of questions, (¶0095, Allen et al "associated with each of the parsed column and row labels, individually or in combination, is identified in step 518 and then assigned in step 520 as metadata to each cell within the table." Figure 5b, 520 "Assign Identified Content Types As Metadata ,Associated With Various Cells In The Table")
to generate query responses (¶0034, Allen at al. "The QA system 100 then generates an output response or answer 120 with the final answer and associated confidence and supporting evidence.").
Allen et al. do not explicitly teach categorize store images based on an image type;
(c )the image type the image type being one of a whole image type associated with two or more shelves or a cropped image type.
However, Adato is in the same field of image processing used for retail store. Further, Adato teaches categorize store images (Adato ¶0283 discloses categorizing at least one image using classification algorithms) based on an image type (Adato ¶0823 discloses images or video being taken of a shelving unit or the whole store);
and (c )the image type (Adato ¶0823 discloses images or video being taken of a shelving unit or the whole store), the image type being one of a whole image type associated with two or more shelves (Adato ¶0395 discloses receiving an image of an entire shelf, Fig 23 discloses an image of two or more shelves, Fig 6c discloses an images of two or more shelves), or a cropped image type (Adato ¶0152, ¶0736 discloses the input image being cropped Fig 46A shows the second image being a cropped image of a single shelf).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen to include categorizing store images based on the type of image and the entire or cropped image pertaining the number of shelves in the image as taught by Adato, to make the invention a method that could use the question answer machine learning model that was created to answer questions about the store observation images based on the number of shelves in the image; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that could be used on store observation images since there is a need for an continuous monitoring of dynamically changing product displays, and to increase productivity, and to provide a dynamic solution that will automatically monitor retail spaces. (Adato, ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Allen and Adato in combination do not explicitly disclose obtain metadata associated, select ones of the store dictionaries for use based on the respective metadata select ones of the store dictionaries for use based on the respective metadata obtain first metadata associated with a first set of questions and second metadata associated with a second set train a machine-learning model based on question-answer pairs the question-answer pairs generated by;
selecting one of the first set of questions or the second set of questions and (b) the second metadata associated with the second set of questions associated with a respective one of the first metadata or the second metadata,
generating question-answer pairs based on the selected ones of the set of the store dictionaries and the selected one of the first set of questions or the second set of questions
and
execute the trained machine-learning model.
Wang is in the same field of processing search queries. Further, Wang teaches obtain metadata associated (Wang Fig 8, ¶0086, and discloses extracting metadata from the acquired text documents which the examiner interprets as equivalent to dictionaries)
select ones of the store dictionaries (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) for use based on the respective metadata (Wang Fig 8, ¶0086, and discloses extracting metadata from the acquired text documents which the examiner interprets as equivalent to dictionaries)
obtain first metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with a first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) and second metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with a second set (Wang ¶0020 and Fig 18 disclose a second set of questions)
train a machine-learning model based on question-answer pairs (Wang Fig 16 discloses training a machine learning model based off question answer pairs), the question-answer pairs generated by (Wang Fig 19 discloses the generation of question and answer pairs):
selecting one (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
and (b) the second metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
associated with a respective one of the first metadata (Wang ¶0047-¶0048 disclose fields associated with each query) or the second metadata (Wang ¶0047-¶0048 disclose fields associated with each query),
generating question-answer pairs (Wang Fig 19 discloses the generation of question and answer pairs)based on the selected ones of the set of the store dictionaries (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) and the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions)or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
execute the trained machine-learning model (Wang ¶0073 disclose performing a search query using the ML model).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen in view of Adato to include the first and second set of questions associated with respective metadata as taught by Wang, to make the invention a method that could use the question answer machine learning model trained on question answer pairs to answer questions about the store observation images; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that could be used on store observation images since there is a need to enables users to intuitively search unstructured data using natural language, that returns specific and personalized answers to questions, giving end users an experience that comes closer to interacting with a human expert. (Wang, ¶0026).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 5, Allen in view of Adato in further view of Wang teaches the at least one non-transitory machine-readable medium as defined in claim 1, wherein the machine-readable instructions, are to cause the machine to mark the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions) for association (¶0093, Allen et al. "A lower initial ranking score (e.g., score=0.5) is then assigned to each QA pair generated using a template that is based on a question referenced to or created from the extracted portion of unstructured text.") with ones of the categorized store images (Adato ¶0283 discloses categorizing at least one image taken from the store using classification algorithms). See rationale for Claim 1, its parent claim.
Regarding Claim 6, Allen in view of Adato in further view of Wang teaches the at least one non-transitory machine-readable medium as defined in claim 1, wherein the machine-readable instructions, are to cause the machine to generate the question-answer pairs (Wang Fig 19 discloses the generation of question and answer pairs) by determining natural language variations (¶0021, Allen et al. "The Named Entity subsystem 112 receives and processes each question 111 by using natural language processing (NLP) to analyze each question and extract question topic information contained in the question") of the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions). See rationale for Claim 1, its parent claim.
Regarding Claim 7, Allen in view of Adato in further view of Wang teaches the at least one non-transitory machine-readable medium as defined in claim 6, wherein the
the selected one (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions) is included in a store observation dataset (¶0024 Allen et al. discloses big datasets coming from a variety of sources) is associated with the store dictionaries (¶0094, Allen et al "followed by the receipt of a corpus of human-readable text, such as a collection of documents, in step 504." Figure 5a, 504 "Receive Corpus Of Human-Readable Text"). See rationale for Claim 1, its parent claim.
Regarding Claim 9, Allen et al. teaches an apparatus to generate query responses comprising (¶0037, Allen et al. "An illustrative example of an information processing system showing an exemplary processor and various components commonly accessed by the processor is shown in Fig. 2"):
interface circuitry (¶0027, Allen et al. discloses "likewise include potentially any other computing device elements generally known in the art including buses, storage devices, communication interfaces, and the like");
machine readable instructions (¶0015, Allen et al. "Computer readable program instructions for carrying out operations of the present invention"); and
to be programmed by the machine readable instructions (¶0012, Allen et al. "Furthermore, aspects of the present invention may take the form of computer program product embodied in a computer readable storage medium, or media, having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.") to:
with respective store dictionaries (¶0094, Allen et al "followed by the receipt of a corpus of human-readable text, such as a collection of documents, in step 504." Figure 5a, 504 "Receive Corpus Of Human-Readable Text");
of questions (¶0095, Allen et al "associated with each of the parsed column and row labels, individually or in combination, is identified in step 518 and then assigned in step 520 as metadata to each cell within the table." Figure 5b, 520 "Assign Identified Content Types As Metadata ,Associated With Various Cells In The Table");
based on (a) the first metadata associated with the first set of questions, (¶0095, Allen et al "associated with each of the parsed column and row labels, individually or in combination, is identified in step 518 and then assigned in step 520 as metadata to each cell within the table." Figure 5b, 520 "Assign Identified Content Types As Metadata ,Associated With Various Cells In The Table")
based on the question-answer pairs (¶0102, Allen et al. "The ranked QA pairs are then used in step 572 to train a QA system" ¶0049, Allen et al. "As used herein, ground truth broadly refers to a set of question-answer (QA) pairs used to train a machine learning system, such as a QA system, where each question of an associated QA pair has a corresponding correct answer."); and
generate query (¶0034, Allen at al. "The QA system 100 then generates an output response or answer 120 with the final answer and associated confidence and supporting evidence.").
Allen et al. do not explicitly teach categorize store images based on an image type the image type being one of a whole image type associated with two or more shelves, or a cropped image type (c) the image type.
However, Adato is in the same field of image processing used for retail store. Further, Adato teaches categorize store images (Adato ¶0283 discloses categorizing at least one image using classification algorithms) based on an image type (Adato ¶0823 discloses images or video being taken of a shelving unit or the whole store) the image type being one of a whole image type associated with two or more shelves (Adato ¶0395 discloses receiving an image of an entire shelf, Fig 23 discloses an image of two or more shelves, Fig 6c discloses an images of two or more shelves), or a cropped image type (Adato ¶0152, ¶0736 discloses the input image being cropped Fig 46A shows the second image being a cropped image of a single shelf) (c )the image type (Adato ¶0823 discloses images or video being taken of a shelving unit or the whole store).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen to include categorizing store images based on the type of image and the entire or cropped image pertaining the number of shelves in the image as taught by Adato, to make the invention a method that could use the question answer machine learning model that was created to answer questions about the store observation images based on the number of shelves in the image; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that could be used on store observation images since there is a need for an continuous monitoring of dynamically changing product displays, and to increase productivity, and to provide a dynamic solution that will automatically monitor retail spaces. (Adato, ¶0003).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Allen and Adato in combination do not explicitly disclose at least one processor circuit,
obtain metadata associated for use based on the respective metadata,
obtain first metadata associated with a first set of questions and second metadata associated with a second set;
select one of the first set of questions or the second set of questions and (b)the second metadata with the second set of questions associated with a respective one of the first metadata or the second metadata,
generate question-answer pairs based on the selected ones of the set of the store dictionaries and the selected one of the first set of questions or the second set of questions
train a machine-learning model based on using the trained machine-learning model
Wang is in the same field of processing search queries. Further, Wang teaches at least one processor circuit (Wang ¶0136-¶0137 discloses the use of processor in hardware such as RAM which a person of ordinary skill in the art would understand to have circuitry)
obtain metadata associated (Wang Fig 8, ¶0086, and discloses extracting metadata from the acquired text documents which the examiner interprets as equivalent to dictionaries)
for use based on the respective metadata (Wang Fig 8, ¶0086, and discloses extracting metadata from the acquired text documents which the examiner interprets as equivalent to dictionaries)
obtain first metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with a first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) and second metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with a second set (Wang ¶0020 and Fig 18 disclose a second set of questions)
select one (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
and (b)the second metadata (Wang ¶0047-¶0048 disclose fields associated with each query) associated with the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
associated with a respective one of the first metadata (Wang ¶0047-¶0048 disclose fields associated with each query) or the second metadata (Wang ¶0047-¶0048 disclose fields associated with each query),
generate question-answer pairs (Wang Fig 19 discloses the generation of question and answer pairs)based on the selected ones of the set of the store dictionaries (Wang Fig 15 and ¶102 discloses selecting a proper subset of data) and the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions)or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions)
train a machine-learning model (Wang Fig 16 discloses training a machine learning model based off question answer pairs),
based on using the trained machine-learning model (Wang ¶0073 disclose performing a search query using the ML model).
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen in view of Adato to include the first and second set of questions associated with respective metadata as taught by Wang, to make the invention a method that could use the question answer machine learning model trained on question answer pairs to answer questions about the store observation images; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that could be used on store observation images since there is a need to enables users to intuitively search unstructured data using natural language, that returns specific and personalized answers to questions, giving end users an experience that comes closer to interacting with a human expert. (Wang, ¶0026).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Regarding Claim 13, Allen in view of Adato in further view of Wang teaches the apparatus as defined in claim 9, wherein one or more of the at least one processor circuit (¶0015, Allen et al. " electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program") is to identify
the image type (Adato ¶0783 discloses the system identifying image data) as the whole image type (Adato ¶0395 discloses receiving an image of an entire shelf) or the cropped image type, the cropped image type associated with a single retail shelf(Adato ¶0152, ¶0736 discloses the input image being cropped Fig 46A shows the second image being a cropped image of a single shelf) .See rationale for Claim 1, its parent claim.
Regarding Claim 14, Allen in view of Adato in further view of Wang teaches the apparatus as defined in claim 9, wherein one or more of the at least one processor circuit (¶0015, Allen et al. " electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program") is to mark the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions) (¶0093, Allen et al. "A lower initial ranking score (e.g., score=0.5) is then assigned to each QA pair generated using a template that is based on a question referenced to or created from the extracted portion of unstructured text.") with ones of the store images (Adato ¶0283 discloses categorizing at least one image taken from the store using classification algorithms).See rationale for Claim 1, its parent claim.
Regarding Claim 15, Allen in view of Adato in further view of Wang teaches the apparatus as defined in claim 9, wherein one or more of the at least one processor circuit (¶0015, Allen et al. " electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program") is to generate the question-answer pairs by determining natural language variations of the selected one of the first set of questions (Wang ¶0019 and Fig 17 disclose a first set of questions) or the second set of questions (Wang ¶0020 and Fig 18 disclose a second set of questions) (¶0021, Allen et al. "The Named Entity subsystem 112 receives and processes each question 111 by using natural language processing (NLP) to analyze each question and extract question topic information contained in the question"). See rationale for Claim 1, its parent claim.
Claim 8 is rejected under 35 U.S.C. 103 as obvious over Allen in view of Adato in view of Wang in further view of Wang et al (US Patent Pub US 2018/0314689 A1 hereafter referred to as Wang 2).
Regarding Claim 8, Allen in view of Adato in view of Wang the at least one non-transitory machine-readable medium (¶0015, Allen et al. "Computer readable program instructions for carrying out operations of the present invention") as defined in claim 1, obtained from a store observation dataset (Adato ¶0142, ¶discloses a database that contains information about the product placement on the store shelves and other related store observation data) including additional content relative to the store dictionaries (Adato Fig 14 1419 and Fig 17, 1713 discloses updating product models which are specific to the product models which are in the store).
Allen et al. in view of Adato in view of Wang do not explicitly teach wherein the machine-readable instructions are to cause the machine to train the machine-learning model using an updated dictionary and the updated dictionary.
However, Wang 2 is in the same field of image processing. Further, Wang 2 teaches wherein the machine-readable instructions are to cause the machine to train the machine-learning model using an updated dictionary (¶0080, Wang et al. "In these implementations, the models, grammars, dictionaries, ontologies, etc. from a well-developed system can be translated, using machine translation, into the multiple languages that are to be supported by a device. The translated models, grammars, etc. can be refined by additional training and/or machine learning techniques.") the updated dictionary (¶0080, Wang et al. "In these implementations, the models, grammars, dictionaries, ontologies, etc. from a well-developed system can be translated, using machine translation, into the multiple languages that are to be supported by a device. The translated models, grammars, etc. can be refined by additional training and/or machine learning techniques.").
Therefore, it would have been obvious to one having ordinary skill in the art before the effective filing date of the claimed invention to modify the invention of Allen in view of Adato in view of Wang to include a store dictionary that is updated as taught by Wang 2, to make the invention a method that could use the question answer machine learning model that was created to expand its range using the store dictionary; thus, one of ordinary skill in the art would be motivated to combine the references to create a question answer machine model that can utilize the store dictionaries since there is a need for increasing the confidence value for the translated text when the additional verbal input affirms the translated text. (Wang 2, ¶0015).
Thus, the claimed subject matter would have been obvious to a person having ordinary skill in the art before the effective filing date of the claimed invention.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
Any inquiry concerning this communication or earlier communications from the
examiner should be directed to RACHEL ROBERTS whose telephone number is (571)272-6413. The examiner can normally be reached Monday- Friday 7:30am- 5:00pm. 32. Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice. 33. If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Oneal Mistry can be reached on 313-446-4912. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300. 34. Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/RACHEL L ROBERTS/Examiner, Art Unit 2674
/ONEAL R MISTRY/Supervisory Patent Examiner, Art Unit 2674
Prosecution Insights