DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
The present application is being examined under the claims filed 02/27/2023.
Claims 1-20 are pending.
Information Disclosure Statement
The information disclosure statement (IDS) submitted on 02/13/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
The information disclosure statement (IDS) submitted on 09/12/2024 is in compliance with the provisions of 37 CFR 1.97. Accordingly, the information disclosure statement is being considered by the examiner.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1-20 are rejected under 35 U.S.C. 101 for containing an abstract idea without significantly more.
Regarding Claim 1:
Step 1 – Is the claim to a process, machine, manufacture, or composition of matter?
Yes, the claim is to a process.
Step 2A – Prong 1 – Does the claim recite an abstract idea, law of nature, or natural phenomenon?
Yes, the claim recites the abstract ideas of:
decoding, by the scanner device, a first barcode represented in the first image data — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because a barcode can be decoded in the human mind or by a human using pen and paper as a tool.
determining a first item template associated with the first barcode, the first item template comprising first identifier data identifying the first item from among other items and first region- of-interest data specifying a first region-of-interest of the first item — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to performing a judgement of items belonging in a template which could be performed in the human mind or by a human using pen and paper.
determining, by a first machine learning model, that the second image data corresponds to the first identifier data identifying the first item — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to evaluating input data using a set of known rules.
and generating first data indicating that the first barcode is matched with the first item — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to performing a judgement of matched data to determine data field parameters.
Step 2A – Prong 2 – Does the claim recite additional elements that integrate the judicial exception into a practical application?
No, the claim does not recite additional elements that integrate the judicial exception into a practical application. The additional elements:
A method comprising: capturing, by a scanner device comprising an image sensor, first image data representing at least a portion of a first item — This limitation is directed to mere data gathering and outputting which has been recognized by the courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) as insignificant extra-solution activity (see MPEP 2106.05(g)).
generating second image data comprising the first region-of-interest of the first image data — This limitation is directed to mere instructions to apply a judicial exception. Using image generation to apply a judicial exception (see MPEP 2106.05(f)) is insufficient to integrate the judicial exception into a practical application. Even if the image generation is implemented on a generic computer (see MPEP 2106.05(f)(2), 2106.04(d)), the limitation does not integrate the judicial exception into a practical application.
Step 2B – Does the claim recite additional elements that amount to significantly more than the abstract idea itself?
No, the claim does not recite additional elements which amount to significantly more than the abstract idea itself. The additional elements as identified in step 2A prong 2:
A method comprising: capturing, by a scanner device comprising an image sensor, first image data representing at least a portion of a first item — This limitation is recited at a high level of generality and amounts to mere data gathering of electronically scanning or extracting data from a physical document, which is well-understood, routine, and conventional activity (see MPEP 2106.05(d) II.), which cannot amount to significantly more than the judicial exception.
generating second image data comprising the first region-of-interest of the first image data — Mere instructions to apply a judicial exception (see MPEP 2106.05(f)) and using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
Regarding Claim 2
Claim 2 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 2:
wherein the first machine learning model comprises a convolutional neural network classifier or visual transformer classifier trained to classify a given item based on an image of a predefined region-of-interest of the given item — This limitation is directed to merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) as it merely limits the field of the machine learning model to a particular kind of model.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
wherein the first machine learning model comprises a convolutional neural network classifier or visual transformer classifier trained to classify a given item based on an image of a predefined region-of-interest of the given item — Merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) cannot amount to significantly more than the judicial exception.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 3
Claim 3 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 1:
further comprising: generating, by the first machine learning model, a first vector representing the second image data — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to evaluating data using known rules to come up with a list of numbers.
comparing the first vector to a plurality of item vectors stored in a data store — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging data against other data to make comparisons.
determining a second vector among the plurality of item vectors based at least in part on a first distance metric used to determine a distance between the first vector and the second vector
and determining that the second vector is associated with the first identifier data in the first item template, wherein the determination that the second image data corresponds to the first identifier data is made based at least in part on the second vector being associated with the first identifier data — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging data against other data to make comparisons.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 4
Claim 4 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 1:
determining, using an object detector, a first bounding box around the first barcode in the first image data — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to evaluating an image to visually determine where a barcode is located in the image (e.g. by drawing a box around the barcode using pen and paper as a tool).
determining a first size of the first bounding box — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging the size of a bounding box (e.g. calculating an area of a rectangle).
determining a first orientation of the first bounding box — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging a rectangle’s orientation to come up with an orientation angle (e.g. looking at a rectangle to determine it is oriented at a 45 degree angle).
determining a second size of a second barcode associated with the first region-of-interest data of the first item template — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging the size of a bounding box (e.g. calculating an area of a rectangle).
and determining a ratio between the first size and the second size — This limitation is directed to the abstract idea of a mathematical process, and mathematical calculations in particular (MPEP 2106.04(a)(2) I. C.). The claim describes the mathematical calculation of dividing two numbers in words.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 5
Claim 5 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 4 which included an abstract idea (see rejection for claim 4). The claim recites the additional limitations:
Step 2A Prong 1:
further comprising determining the first region-of-interest of the first image data based at least in part by: resizing a second bounding box corresponding to the first region-of-interest of the first item in the first item template using the ratio — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to evaluating a bounding box based on a calculated ratio to determine a new size.
and applying the re-sized second bounding box to the first image data — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging where a new bounding box should be placed on an image (e.g. drawing a new bounding box on a picture using pen/paper).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 6
Claim 6 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 1:
decoding, by the scanner device, a second barcode represented in the third image data — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because a barcode can be decoded in the human mind or by a human using pen and paper as a tool.
determining a second item template associated with the second barcode, the second item template comprising second identifier data identifying the second item from among other items and second region-of-interest data specifying a second region-of-interest of the second item that includes the second barcode and a second non-barcode portion of the second item — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to performing a judgement of items belonging in a template which could be performed in the human mind or by a human using pen and paper.
determining, by the first machine learning model, that the fourth image data is mismatched with respect to the second barcode — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to evaluating input data using a set of known rules.
and generating first output data indicating that the second barcode is mismatched with respect to the second item — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to performing a judgement of mismatched data to determine data field parameters.
Step 2A Prong 2:
capturing, by the scanner device, third image data representing at least a portion of a second item — This limitation is directed to mere data gathering and outputting which has been recognized by the courts (as per Ultramercial, 772 F.3d at 715, 112 USPQ2d at 1754) as insignificant extra-solution activity (see MPEP 2106.05(g)).
generating fourth image data comprising the second region-of-interest of the third image data — This limitation is directed to mere instructions to apply a judicial exception. Using image generation to apply a judicial exception (see MPEP 2106.05(f)) is insufficient to integrate the judicial exception into a practical application. Even if the image generation is implemented on a generic computer (see MPEP 2106.05(f)(2), 2106.04(d)), the limitation does not integrate the judicial exception into a practical application.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
capturing, by the scanner device, third image data representing at least a portion of a second item — This limitation is recited at a high level of generality and amounts to mere data gathering of electronically scanning or extracting data from a physical document, which is well-understood, routine, and conventional activity (see MPEP 2106.05(d) II.), which cannot amount to significantly more than the judicial exception.
generating fourth image data comprising the second region-of-interest of the third image data — Mere instructions to apply a judicial exception (see MPEP 2106.05(f)) and using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 7
Claim 7 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 1:
generating second identifier data identifying the second item from among other items — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to an evaluation of data.
Step 2A Prong 2:
generating third image data representing a second region-of-interest of a second item, the second region-of-interest representing a second barcode of the second item and at least a second non-barcode portion of the second item — This limitation is directed to mere instructions to apply a judicial exception. Using generic image processing to apply a judicial exception (see MPEP 2106.05(f)) is insufficient to integrate the judicial exception into a practical application. Even if the generic image processing is implemented on a generic computer (see MPEP 2106.05(f)(2), 2106.04(d)), the limitation does not integrate the judicial exception into a practical application.
generating a first training instance comprising the third image data and the second identifier data — This limitation is directed to mere instructions to apply a judicial exception. Using generic data structures to apply a judicial exception (see MPEP 2106.05(f)) is insufficient to integrate the judicial exception into a practical application. Even if the generic data structures are implemented on a generic computer (see MPEP 2106.05(f)(2), 2106.04(d)), the limitation does not integrate the judicial exception into a practical application.
and training the first machine learning model to classify items using a training dataset comprising the first training instance — This limitation is directed to mere instructions to apply a judicial exception. Using generic machine learning training to apply a judicial exception (see MPEP 2106.05(f)) is insufficient to integrate the judicial exception into a practical application. Even if the generic machine learning training is implemented on a generic computer (see MPEP 2106.05(f)(2), 2106.04(d)), the limitation does not integrate the judicial exception into a practical application.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
generating third image data representing a second region-of-interest of a second item, the second region-of-interest representing a second barcode of the second item and at least a second non-barcode portion of the second item — Mere instructions to apply a judicial exception (see MPEP 2106.05(f)) and using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
generating a first training instance comprising the third image data and the second identifier data — Mere instructions to apply a judicial exception (see MPEP 2106.05(f)) and using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
and training the first machine learning model to classify items using a training dataset comprising the first training instance — Mere instructions to apply a judicial exception (see MPEP 2106.05(f)) and using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 8
Claim 8 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 2:
wherein the first region-of-interest of the first item includes the first barcode and a non-barcode portion of the first item — This limitation is directed to merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) as it merely limits the field of what the region of interest contains.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
wherein the first region-of-interest of the first item includes the first barcode and a non-barcode portion of the first item — Merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) cannot amount to significantly more than the judicial exception.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 9
Claim 9 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 2:
wherein the first item template represents at least one of a contextual or a geometric relationship between the first barcode and the first region-of-interest of the first item — This limitation is directed to merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) as it merely limits the field of the item template.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
wherein the first item template represents at least one of a contextual or a geometric relationship between the first barcode and the first region-of-interest of the first item —Merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) cannot amount to significantly more than the judicial exception.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 10
Claim 10 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 2:
wherein the first item template further comprises data representing a barcode type of the first barcode — This limitation is directed to merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) as it merely limits the field of the first item template.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
wherein the first item template further comprises data representing a barcode type of the first barcode — Merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) cannot amount to significantly more than the judicial exception.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 11
Claim 11 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 1 which included an abstract idea (see rejection for claim 1). The claim recites the additional limitations:
Step 2A Prong 2:
wherein the first item template further comprises: a template image of the first region-of-interest of the first item; and at least one of coordinate data representing a location in the template image of the first barcode, orientation data representing an orientation in the template image of the first barcode, or size data representing a size of the first barcode in the template image —This limitation is directed to merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) as it merely limits the field of the first item template.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
The additional elements as identified in step 2A prong 2:
wherein the first item template further comprises: a template image of the first region-of-interest of the first item; and at least one of coordinate data representing a location in the template image of the first barcode, orientation data representing an orientation in the template image of the first barcode, or size data representing a size of the first barcode in the template image —Merely limiting a judicial exception to a particular field of use (see MPEP 2106.05(h)) cannot amount to significantly more than the judicial exception.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 12
Independent claim 12 is a computer system claim corresponding to method claim 1, which was directed to an abstract idea, therefore the same rejection and rationale applies. The only differences are that claim 12 does not include the scanner device and recites the following additional elements treated under step 2A prong 2 and step 2B:
Step 2A Prong 2:
A system comprising: an image sensor; at least one processor; and non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to — This limitation is directed to merely applying an abstract idea using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.04(d)).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2.
Step 2B:
A system comprising: an image sensor; at least one processor; and non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to — Using a generic computer as a tool (see MPEP 2106.05(f)(2), 2106.05(d)) cannot amount to significantly more than the judicial exception itself.
Thus, the claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 13
Dependent claim 13 is a computer system claim corresponding to method claim 2, which was directed to an abstract idea, therefore the same rejection and rationale applies.
Regarding Claim 14
Dependent claim 14 is a computer system claim corresponding to method claim 3, which was directed to an abstract idea, therefore the same rejection and rationale applies.
Regarding Claim 15
Claim 15 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 12 which included an abstract idea (see rejection for claim 12). The claim recites the additional limitations:
Step 2A Prong 1:
the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine, using an object detector, a first bounding box around the first barcode in the first image data — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to evaluating an image to visually determine where a barcode is located in the image (e.g. by drawing a box around the barcode using pen and paper as a tool).
determine a first orientation of the first bounding box — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging a rectangle’s orientation to come up with an orientation angle (e.g. looking at a rectangle to determine it is oriented at a 45 degree angle).
determine a second orientation of the barcode associated with the first region-of-interest data of the first item template — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging a rectangle’s orientation to come up with an orientation angle (e.g. looking at a rectangle to determine it is oriented at a 45 degree angle).
and determine an amount of rotation between the first orientation and the second orientation — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging a first and second value by making comparisons.
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 16
Claim 16 is rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The claim is dependent on claim 15 which included an abstract idea (see rejection for claim 15). The claim recites the additional limitations:
Step 2A Prong 1:
the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: re-orient a second bounding box corresponding to the first region-of-interest of the first item in the first item template based on the amount of rotation — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to evaluating a bounding box to determine a new orientation based on an amount of rotation.
and apply the re-oriented second bounding box to the first image data — This limitation is directed to the abstract idea of a mental process (including an observation, evaluation, judgement, opinion) which can be performed by the human mind, or by a human using pen and paper (see MPEP 2106.04(a)(2) III. C.). The limitation is directed to a mental process because it amounts to judging where a new bounding box should be placed on an image (e.g. drawing a new bounding box on a picture using pen/paper).
Thus, the judicial exception is not integrated into a practical application (see MPEP 2106.04(d) I.), failing step 2A prong 2. The claim does not include additional elements that are sufficient to amount to significantly more than the judicial exception under step 2B.
Regarding Claim 17
Dependent claim 17 is a computer system claim corresponding to method claim 6, which was directed to an abstract idea, therefore the same rejection and rationale applies.
Regarding Claim 18
Dependent claim 18 is a computer system claim corresponding to method claim 7, which was directed to an abstract idea, therefore the same rejection and rationale applies.
Regarding Claim 19
Independent claim 19 is a method claim corresponding to method claim 1, which was directed to an abstract idea, therefore the same rejection and rationale applies. The only difference is that claim 19 recites receiving first image data instead of capturing, by a scanner device comprising an image sensor. The same rejection and rationale apply.
Regarding Claim 20
Dependent claim 20 is a method claim corresponding to method claim 3, which was directed to an abstract idea, therefore the same rejection and rationale applies.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
Claims 1, 4-6, 8-10, 12, 17, and 19 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Deshmukh et al. (PGPUB no. US20210217129A1) herein referred to as Deshmukh.
Regarding Claim 1
Deshmukh teaches:
A method comprising: capturing, by a scanner device comprising an image sensor, first image data representing at least a portion of a first item;
(paragraph [0057]) “During the scanning operation, the POS scanning application software obtains the output of the scanner, which is comprised of the recognized codes, e.g., GTIN, price change code, or like code.”; (paragraph [0059]) “During check out at the POS terminal, the scanner executes recognition operations on image frames captured while a product package or packages move through its field of view.”
decoding, by the scanner device, a first barcode represented in the first image data;
(paragraph [0058]) “It receives each code from the scanner, in response to the scanner decoding UPC and Digimarc Barcode data carrier during check-out. A processor in the scanner executes firmware instructions loaded from memory to perform these decoding operations.”
determining a first item template associated with the first barcode, the first item template comprising first identifier data identifying the first item from among other items and first region- of-interest data specifying a first region-of-interest of the first item;
(paragraph [0201]) “The scanner may also include a recognition unit that implements an image recognition method for identifying a product in a store's inventory as well as product labels, such as price change labels. In such a system, reference image feature sets of each product are stored in a database[*Examiner notes: first template] of the scanner's memory and linked to an item identifier for a product and/or particular label (e.g., price change label). The recognition unit extracts corresponding features from an image frame and matches them against the reference feature sets to detect a likely match. If the match criteria are satisfied, the recognition unit returns an item identifier[*Examiner notes: identifier data] to the controller. The recognition unit may also return spatial information, such as position, bounding box[*Examiner notes: region of interest data], shape or other geometric parameters for a recognized item to enable the controller to detect whether a code from another recognition unit is from the same object”; Figure 15
PNG
media_image1.png
227
360
media_image1.png
Greyscale
generating second image data comprising the first region-of-interest of the first image data;
(paragraph [0099]) "FIG. 6 is diagram illustrating a software modules 160, 162 that operate on a sequence of image frames 164 to detect and extract digital payloads from images of objects within the frames."; (paragraph [0201]) “The recognition unit may also return spatial information, such as position, bounding box, shape or other geometric parameters for a recognized item to enable the controller to detect whether a code from another recognition unit is from the same object.”
determining, by a first machine learning model, that the second image data corresponds to the first identifier data identifying the first item;
(paragraph [0201]) “The scanner may also include a recognition unit that implements an image recognition method for identifying a product in a store's inventory as well as product labels, such as price change labels. In such a system, reference image feature sets of each product are stored in a database of the scanner's memory and linked to an item identifier for a product and/or particular label (e.g., price change label). The recognition unit extracts corresponding features from an image frame and matches them against the reference feature sets to detect a likely match […] The recognition unit may also return spatial information, such as position, bounding box, shape or other geometric parameters for a recognized item to enable the controller to detect whether a code from another recognition unit is from the same object.”
and generating first data indicating that the first barcode is matched with the first item.
(paragraph [0201]) “The recognition unit extracts corresponding features from an image frame and matches them against the reference feature sets to detect a likely match. If the match criteria are satisfied, the recognition unit returns an item identifier to the controller[*Examiner notes: first data indicating match].”
Regarding Claim 4
Deshmukh teaches:
The method of claim 1
(see rejection of claim 1)
further comprising: determining, using an object detector, a first bounding box around the first barcode in the first image data;
(paragraph [0057]) “During the scanning operation, the POS scanning application software obtains the output of the scanner, which is comprised of the recognized codes, e.g., GTIN, price change code, or like code.”
determining a first size of the first bounding box
(paragraph [0258]) "A portion of image data 500 is extracted or obtained 561 that corresponds to the area bounded by the minimum bounding box. For example, the corresponding pixels that are within the area (e.g., the corresponding spatial locations) identified by the minimum bounding box are obtained for further evaluation."
determining a first orientation of the first bounding box; determining a second size of a second barcode associated with the first region-of-interest data of the first item template;
(para [0248]) "For example, the bounding box (and its image contents) can be rotated such that one of its edges is horizontal to an image plane. And the image data within the candidate contour can be resized, e.g., according to the sizing of previously stored templates."
and determining a ratio between the first size and the second size
[*Examiner notes: The broadest reasonable interpretation of a ratio is any comparison between numbers, including the ranking comparisons taught by Deshmukh]; (paragraph [0257]) "For example, if a target icon has an aspect ratio near 1, candidate contours can be ranked according to their determined aspect ratios, with the closest aspect ratio to 1 being evaluated first, and the second closest being evaluated next, and then so on. In another example, the candidate contours are ranked according to their minimum bounding box area (or an area calculated for the closed contour), with the largest area first, and the smallest area last."; see also para [0248] and [0258]).
Regarding Claim 5
Deshmukh teaches:
The method of claim 4
(see rejection of claim 4)
further comprising determining the first region-of-interest of the first image data based at least in part by: resizing a second bounding box corresponding to the first region-of-interest of the first item in the first item template using the ratio; and applying the re-sized second bounding box to the first image data.
[*Examiner notes: The block is resized from its original size to the template size and applied to the extracted image data. This involves the ratio between the original size and the template size. ](paragraph [0258]) “This orientation process helps the icon matching be more rotation invariant relative to an un-rotated block. The block can then be resized 563 to match or approximate the size of the template(s).”
PNG
media_image2.png
689
311
media_image2.png
Greyscale
Regarding Claim 6
Deshmukh teaches:
The method of claim 1
(see rejection claim 1)
further comprising: capturing, by the scanner device, third image data representing at least a portion of a second item
(paragraph [0068]) "Fixed scanners are designed to be integrated within a check-out station, at which the operator or a conveyor moves items in the field of the scanner's image capture system."; (see also paragraph [0057) and [0059])
decoding, by the scanner device, a second barcode represented in the third image data
(paragraph [0058]) “It receives each code from the scanner, in response to the scanner decoding UPC and Digimarc Barcode data carrier during check-out. A processor in the scanner executes firmware instructions loaded from memory to perform these decoding operations.”
determining a second item template associated with the second barcode, the second item template comprising second identifier data identifying the second item from among other items and second region-of-interest data specifying a second region-of-interest of the second item that includes the second barcode and a second non-barcode portion of the second item
(paragraph [0201]) “The scanner may also include a recognition unit that implements an image recognition method for identifying a product in a store's inventory as well as product labels, such as price change labels. In such a system, reference image feature sets of each product are stored in a database of the scanner's memory and linked to an item identifier for a product and/or particular label (e.g., price change label). The recognition unit extracts corresponding features from an image frame and matches them against the reference feature sets to detect a likely match. If the match criteria are satisfied, the recognition unit returns an item identifier to the controller. The recognition unit may also return spatial information, such as position, bounding box, shape or other geometric parameters for a recognized item to enable the controller to detect whether a code from another recognition unit is from the same object”; () “Consider FIG. 15, where an object (e.g., representing one face of a retail package) includes artwork, text, and various machine-readable symbologies. In the illustrated example, the artwork includes castles, sundial, shields, knight/horse, scenery, etc. The text includes “VALIANT”, “For the courage to get deep down clean”, “ICON Label”, etc. And a 1D barcode and a 2D barcode.”; Figure 15
PNG
media_image3.png
403
871
media_image3.png
Greyscale
generating fourth image data comprising the second region-of-interest of the third image data
(paragraph [0099]) "FIG. 6 is diagram illustrating a software modules 160, 162 that operate on a sequence of image frames 164 to detect and extract digital payloads from images of objects within the frames."; (paragraph [0201]) “The recognition unit may also return spatial information, such as position, bounding box, shape or other geometric parameters for a recognized item to enable the controller to detect whether a code from another recognition unit is from the same object.”
determining, by the first machine learning model, that the fourth image data is mismatched with respect to the second barcode
(para [0265]) - "If the number of remaining objects is equal to (or greater than) m, flow moves on to template correlation 566 and comparison with threshold 567 as discussed above with reference to FIG. 20C. If not, it is determined that the candidate does not match the target icon."; (see also paragraph [0072] and [0201]);
and generating first output data indicating that the second barcode is mismatched with respect to the second item
(paragraph [0215]) "To avoid false positives, a "no match" output is produced if the distance score for the best match is close- e.g., 25 %-to the distance score for the next-best match.”
Regarding Claim 8
Deshmukh teaches:
The method of claim 1
(see rejection of claim 1)
wherein the first region-of-interest of the first item includes the first barcode and a non-barcode portion of the first item.
(paragraph [0201]) “The scanner may also include a recognition unit that implements an image recognition method for identifying a product in a store's inventory as well as product labels, such as price change labels. In such a system, reference image feature sets of each product are stored in a database of the scanner's memory and linked to an item identifier for a product and/or particular label (e.g., price change label). The recognition unit extracts corresponding features from an image frame and matches them against the reference feature sets to detect a likely match. If the match criteria are satisfied, the recognition unit returns an item identifier to the controller. The recognition unit may also return spatial information, such as position, bounding box, shape or other geometric parameters for a recognized item to enable the controller to detect whether a code from another recognition unit is from the same object”; () “Consider FIG. 15, where an object (e.g., representing one face of a retail package) includes artwork, text, and various machine-readable symbologies. In the illustrated example, the artwork includes castles, sundial, shields, knight/horse, scenery, etc. The text includes “VALIANT”, “For the courage to get deep down clean”, “ICON Label”, etc. And a 1D barcode and a 2D barcode.”; Figure 15
PNG
media_image3.png
403
871
media_image3.png
Greyscale
Regarding Claim 9
Deshmukh teaches:
The method of claim 1
(see rejection of claim 1)
wherein the first item template represents at least one of a contextual or a geometric relationship between the first barcode and the first region-of-interest of the first item.
(paragraph [0096]) "Attributes of the item include color (e.g., color histogram) or geometry, such as position, shape, bounding region or other geometric attributes). The attributes may be further submitted to a classifier to classify an item type."; (paragraph [0201) "The recognition unit may also return spatial information, such as position, bounding box, shape or other geometric parameters for a recognized item to enable the controller to detect whether a code from another recognition unit is from the same object."
Regarding Claim 10
Deshmukh teaches:
The method of claim 1
(see rejection of claim 1)
wherein the first item template further comprises data representing a barcode type of the first barcode.
(para [0066]-[0067] - "For a fixed price code detected as the OB of FIG. 3, an inner barcode detected in the waiting period is ignored. For a discount code, the inner barcode in the waiting period is reported ... The details of the implementation vary with the hardware and software configuration of the scanner, as well as the type of codes and recognition processes employed within the scanner."; para [0129] - "For example, a GTIN in a barcode of one type reported from one recognition unit may agree with a GTIN in a different symbology reported from another recognition unit."; see also para [0061] and [0201]).
Regarding Claim 12
Claim 12 is a computer system claim corresponding to method claim 1. The only differences are that claim 12 does not include the scanner device and does include a computer system:
Deshmukh teaches:
A system comprising: an image sensor; at least one processor; and non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to
(paragraph [0049]) “A system comprising: an image sensor; at least one processor; and non-transitory computer-readable memory storing instructions that, when executed by the at least one processor, are effective to:”
The remaining limitations of the claim are taught by the rejection of claim 1.
Regarding Claim 17
Claim 17 is a computer system claim corresponding to method claim 6. The difference is that claim 6 recites a computer system as taught in the rejection of claim 12 above. The remaining limitations of the claim are taught by the rejection of claim 6.
Regarding Claim 19
Claim 19 is a method claim corresponding to method claim 1. The only difference is that claim 19 recites receiving first image data instead of capturing, by a scanner device comprising an image sensor. The same rejection and rationale applies.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claims 2, 7, 13, and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Deshmukh in view of Skaff et al. (PGPUB no. US 20190034864 A1) herein referred to as Skaff.
Regarding Claim 2
Deshmukh teaches:
The method of claim 1
(see rejection of claim 1)
Deshmukh does not teach:
wherein the first machine learning model comprises a convolutional neural network classifier or visual transformer classifier trained to classify a given item based on an image of a predefined region-of-interest of the given item
However, Skaff teaches:
wherein the first machine learning model comprises a convolutional neural network classifier or visual transformer classifier trained to classify a given item based on an image of a predefined region-of-interest of the given item
(paragraph [0046]) “Segmentation can be improved by training classifiers on annotated training data sets, where bounding boxes are manually drawn around products. Training can be performed with supervised or unsupervised machine learning, deep learning, or hybrid machine and deep learning techniques, including but not limited to convolutional neural networks.”
Deshmukh, Skaff, and the instant application are analogous because they are all directed to image classification.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the barcode scanning and image recognition of Deshmukh with the convolutional neural network of Skaff because (Skaff paragraph [0047]) “Segmentation can be improved by training classifiers on annotated training data sets, where bounding boxes are manually drawn around products. Training can be performed with supervised or unsupervised machine learning, deep learning, or hybrid machine and deep learning techniques, including but not limited to convolutional neural networks.”
Regarding Claim 7
Deshmukh teaches:
The method of claim 1
(see rejection of claim 1)
Skaff teaches:
further comprising: generating third image data representing a second region-of-interest of a second item, the second region-of-interest representing a second barcode of the second item and at least a second non-barcode portion of the second item;
(paragraph [0045]) “Segmented images can assist in defining a product bounding box that putatively identifies a product facing.”; (paragraph [0098]) “Typically, high resolution images can capture bar codes[*Examiner notes: barcode portion], product names, or other identifiers printed and visible on the product[*Examiner notes: non-barcode portion] box or container.”
generating second identifier data identifying the second item from among other items;
(paragraph [0045]) “Segmented images can assist in defining a product bounding box that putatively identifies a product facing. This information is often necessary to develop a product library[*Examiner notes: identifying the item from among other items].”; (paragraph [0006]) “defining a product bounding box; associating the bounding box to a shelf label to build a training data set;”
generating a first training instance comprising the third image data and the second identifier data; and training the first machine learning model to classify items using a training dataset comprising the first training instance.
(paragraph [0006]) “associating the bounding box to a shelf label to build a training data set; and using the training data set to train a product classifier.”
Deshmukh, Skaff, and the instant application are analogous because they are all directed to image classification.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the barcode scanning and image recognition of Deshmukh with the training taught by Skaff because (Skaff paragraph [0047]) “Segmentation can be improved by training classifiers on annotated training data sets, where bounding boxes are manually drawn around products. Training can be performed with supervised or unsupervised machine learning, deep learning, or hybrid machine and deep learning techniques, including but not limited to convolutional neural networks.”
Regarding Claim 13
Claim 13 is a computer system claim corresponding to method claim 2. The only difference is that claim 13 recites as taught in the rejection of claim 12 above. The remaining limitations of the claim are taught by the rejection of claim 2.
Regarding Claim 18
Claim 18 is a computer system claim corresponding to method claim 7. The only difference is that claim 18 recites as taught in the rejection of claim 12 above. The remaining limitations of the claim are taught by the rejection of claim 7.
Claims 3, 14, and 20 are rejected under 35 U.S.C. 103 as being unpatentable over Deshmukh in view of Srivastava et al. (PGPUB no. US 20240095709 A1).
Regarding Claim 3
Deshmukh teaches:
The method of claim 1
(see rejection of claim 1)
further comprising: generating, by the first machine learning model, a first vector representing the second image data;
(paragraph [0222]) "Such methods extract local features from patches of an image[*Examiner notes: second image data] (e.g., SIFT points), and automatically cluster the features into N groups (e.g., 168 groups)-each corresponding to a prototypical local feature. A vector of occurrence counts[*Examiner notes: first vector] of each of the groups (i.e., a histogram) is then determined, and serves as a reference signature for the image.
Deshmukh does not explicitly teach:
comparing the first vector to a plurality of item vectors stored in a data store; determining a second vector among the plurality of item vectors based at least in part on a first distance metric used to determine a distance between the first vector and the second vector;
However, Srivastava teaches:
comparing the first vector to a plurality of item vectors stored in a data store; determining a second vector among the plurality of item vectors based at least in part on a first distance metric used to determine a distance between the first vector and the second vector;
(paragraph [0140]) “Additionally or alternatively, determining the item identifier can include determining a distance or similarity score[*Examiner notes: based on distance metric] (e.g., similarity metric) between the unknown item's feature vectors (e.g., image and/or geometric feature vectors) and a set of reference feature vectors[*Examiner notes: plurality of item vectors] (e.g., image and/or geometric feature vectors) associated with a set of known item identifiers, and selecting the item identifier associated with the best distance or similarity score[*Examiner notes: second vector] (e.g., smallest distance, furthest distance, most similar, etc.)”; [*Examiner notes: The second vector is the vector with the best distance or similarity score and the item identifier is the associated identifier]
and determining that the second vector is associated with the first identifier data in the first item template, wherein the determination that the second image data corresponds to the first identifier data is made based at least in part on the second vector being associated with the first identifier data.
(paragraph [0140]) “Additionally or alternatively, determining the item identifier can include determining a distance or similarity score (e.g., similarity metric) between the unknown item's feature vectors (e.g., image and/or geometric feature vectors) and a set of reference feature vectors (e.g., image and/or geometric feature vectors) associated with a set of known item identifiers, and selecting the item identifier associated with the best distance or similarity score (e.g., smallest distance, furthest distance, most similar, etc.); an example is shown in FIG. 12 .”
Deshmukh, Srivastava, and the instant application are analogous because they are all directed to image classification.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the barcode scanning and image recognition of Deshmukh with the vectors and distance comparisons taught by Srivastava because (Srivastava paragraph [0026]) “First, the method can improve item segmentation and identification accuracy by leveraging 3D visual data instead of processing only 2D data.”
Regarding Claim 14
Claim 14 is a computer system claim corresponding to method claim 3. The only difference is that claim 14 recites as taught in the rejection of claim 12 above. The remaining limitations of the claim are taught by the rejection of claim 3.
Regarding Claim 20
Claim 20 is a method claim corresponding to method claim 3. The same rejection and rationale applies.
Claim 11 is rejected under 35 U.S.C. 103 as being unpatentable over Deshmukh in view of Larson et al. (PGPUB no. US 20220067436 A1) herein referred to as Larson and Migdal et al. (PGPUB no. US 20150193780 A1) herein referred to as Migdal.
Regarding Claim 11
Deshmukh teaches:
The method of claim 1
(see rejection of claim 1)
Deshmukh does not explicitly teach:
wherein the first item template further comprises: a template image of the first region-of-interest of the first item;
and at least one of coordinate data representing a location in the template image of the first barcode, orientation data representing an orientation in the template image of the first barcode, or size data representing a size of the first barcode in the template image.
However, Larson teaches:
wherein the first item template further comprises: a template image of the first region-of-interest of the first item
(paragraph [0022]) “FIG. 1 shows an embodiment of a system 100 of facilitating computer-vision-based detection of consumer products. The system 100 includes an image collection apparatus 110, which, as will be described below, is generally an apparatus for building a database of reference image data in connection with a wide variety of consumer products 190”; (paragraph [0052]) “. The images in FIG. 4, also referred to as item masks, serve as reference bounding box models for each of the consumer products 190 to facilitate identification of the consumer products 190 using the images generated in the image collection apparatus 110.”
Deshmukh, Larson, and the instant application are analogous because they are all directed to image classification.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the barcode scanning and image recognition of Deshmukh with the template region of interest data of Larson because (Larson paragraph [0074]) “and then correlating the objects detected in the image to the reference image data models, which were generated in the image collection apparatus 110 and are stored in association with identifiers of specific consumer products 190 in the electronic database 140, in order to recognize an identity of the consumer products 190 representing the objects detected in the images of the shelf 185.”
And Migdal teaches:
and at least one of coordinate data representing a location in the template image of the first barcode, orientation data representing an orientation in the template image of the first barcode, or size data representing a size of the first barcode in the template image.
(paragraph [0094]) “Once the barcode is fitted with the bounding box 603 at 902, the projective transform can be obtained by comparing the size and shape of the barcode in the image to the predetermined reference size and shape of the barcode corresponding to the item identifier number. The projective transform is applied to the image at 901 to arrive at the canonical view 903.”
Deshmukh, Larson, Migdal, and the instant application are analogous because they are all directed to image classification.
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to modify the barcode scanning and image recognition of Deshmukh in view of Larson with the barcode template size and shape taught by Migdal because (Migdal paragraph [0094]) “Then the size and shape of the barcode in the image is compared to the predetermined reference size and shape of the barcode corresponding to the item identifier number to calculate the affine transform. The image at 803 shows the image with the affine distortion removed by applying the affine transform to the image 801.” That is, the reference size and shape helps remove distortion of a barcode image.
Claims 15-16 are rejected under 35 U.S.C. 103 as being unpatentable over Deshmukh in view of NPL reference Liu et al. “Learning a Rotation Invariant Detector with Rotatable Bounding Box” herein referred to as Liu.
Regarding Claim 15
Deshmukh teaches:
The system of claim 12
(see rejection of claim 12)
the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: determine, using an object detector, a first bounding box around the first barcode in the first image data;
(paragraph [0057]) “During the scanning operation, the POS scanning application software obtains the output of the scanner, which is comprised of the recognized codes, e.g., GTIN, price change code, or like code.”
determine a first orientation of the first bounding box;
(paragraph [0258]) "A portion of image data 500 is extracted or obtained 561 that corresponds to the area bounded by the minimum bounding box. For example, the corresponding pixels that are within the area (e.g., the corresponding spatial locations) identified by the minimum bounding box are obtained for further evaluation."
determine a second orientation of the barcode associated with the first region-of-interest data of the first item template;
(paragraph [0248]) “The minimum bounding box helps facilitate re-orientation 532 of the candidate contour to resolve image rotation and scale. For example, the bounding box (and its image contents) can be rotated such that one of its edges is horizontal to an image plane. And the image data within the candidate contour can be resized, e.g., according to the sizing of previously stored templates.”
Deshmukh does not explicitly teach:
and determine an amount of rotation between the first orientation and the second orientation.
However, Liu teaches:
and determine an amount of rotation between the first orientation and the second orientation.
(page 3 column 2) “ArIoU between RBox A and B decreases monotonically when their angle difference changes from 0 degree to 90 degrees. The two definitions differ in the behavior when (θA −θB) is near 180 degrees.”
Regarding Claim 16
Deshmukh in view of Liu teaches:
The system of claim 15
(see rejection of claim 15)
And Liu further teaches:
the non-transitory computer-readable memory storing further instructions that, when executed by the at least one processor, are further effective to: re-orient a second bounding box corresponding to the first region-of-interest of the first item in the first item template based on the amount of rotation; and apply the re-oriented second bounding box to the first image data.
(page 4 column 2 above section 3.3) “The angle regression term applies tangent function to adapt to the periodicity of the angle parameter. The minimization of the angle regression term ensures that the correct angle is learned during training.”
It would have been obvious to a person having ordinary skill in the art before the effective filing date of the present invention to combine Deshmukh and Liu for the same reasons given in claim 15 above.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to Ezra J Baker whose telephone number is (703)756-1087. The examiner can normally be reached Monday - Friday 10:00 am - 8:00 pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, David Yi can be reached at (571) 270-7519. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/E.J.B./Examiner, Art Unit 2126
/DAVID YI/Supervisory Patent Examiner, Art Unit 2126
Prosecution Insights