DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Interpretation
Under MPEP 2143.03, "All words in a claim must be considered in judging the patentability of that claim against the prior art." In re Wilson, 424 F.2d 1382, 1385, 165 USPQ 494, 496 (CCPA 1970). As a general matter, the grammar and ordinary meaning of terms as understood by one having ordinary skill in the art used in a claim will dictate whether, and to what extent, the language limits the claim scope. Language that suggests or makes a feature or step optional but does not require that feature or step does not limit the scope of a claim under the broadest reasonable claim interpretation. In addition, when a claim requires selection of an element from a list of alternatives, the prior art teaches the element if one of the alternatives is taught by the prior art. See, e.g., Fresenius USA, Inc. v. Baxter Int’l, Inc., 582 F.3d 1288, 1298, 92 USPQ2d 1163, 1171 (Fed. Cir. 2009).
Claims 2 and 11 recite “at least one of.” Since “at least one of” is disjunctive, any one of the elements found in the prior art is sufficient to reject the claim. While citations have been provided for completeness and rapid prosecution, only one element is required. Because, on balance, it appears the disjunctive interpretation enjoys the most specification support and for that reason the disjunctive interpretation (one of A, B OR C) is being adopted for the purposes of this Office Action. Applicant’s comments and/or amendments relating to this issue are invited to clarify the claim language and the prosecution history.
Claim Rejections - 35 USC § 101
35 U.S.C. 101 reads as follows:
Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and requirements of this title.
Claims 1- 9 are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations, under their broadest reasonable interpretation, cover mental process using images/ drawings (concept performed in a human mind, including as observation, evaluation, judgment, opinion, prediction, etc.), and mathematical calculations for likelihood/ probability (e.g., - P(A) = f / N Where P(A) = Probability of an event (event A) occurring; f = Number of ways an event can occur (frequency); N = Total number of outcomes possible).
This judicial exception is not integrated into a practical application because the steps do not add meaningful limitations to be considered specifically applied to a particular technological problem to be solved. The claims do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be done mentally and no additional features in the claims would preclude them from being performed as such.
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claim 1is directed to an abstract idea as shown below:
STEP 1: Do the claims fall within one of the statutory categories?
YES. Claim 1 is directed to a method, i.e., process.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea?
YES, the claims are directed toward a mental process (i.e., abstract idea).
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
The method in claim 1, for example, comprises a mental process that can be practicably performed in the human mind therefore, an abstract idea.
Claim 1 recites:
accessing batch data…
applying a contained-item identification model…
generating a set of pairs…
identifying a sorting error…
transmitting an alert…
These limitations, as drafted, under their broadest reasonable interpretation, cover performance of the limitations in the mind or by a human. The Examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that “can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[M]ental processes and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same).
As such, a person could present learn a mental model, observe pictures of sorted goods and determine if there is an error with corresponding outline(s) with a degree of error or lack thereof either mentally or using a pen and paper. The mere nominal recitation that the various steps are being executed by a processor (e.g., processing unit) does not take the limitations out of the mental process grouping. Thus, the claims recite a mental process.
If a claim limitation, under its broadest reasonable interpretation, covers performance of a mental step which could be performed with a simple tool such as a pen and paper, then it falls within the “mental steps” grouping of abstract ideas. Accordingly, the claim recites an abstract idea.
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
NO, the claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Thus, Claims 1- 9 do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application.
Thus, since Claim 1 is: (a) directed toward an abstract idea, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, claim 1 is not eligible subject matter under 35 U.S.C 101. Similar analysis is made for the dependent claims 2-9 and the dependent claims are similarly identified as: being directed towards an abstract idea, not reciting additional elements that integrate the judicial exception into a practical application, and not reciting additional elements that amount to significantly more than the judicial exception.
1st Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claims 1-3, 5, 8-12, 14 and 17-20 (all claims except 4, 6-7, 13 and 15-16) are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2025 0091751 A1, (Prasad et al.) in view of US Patent Publication 2021 AAAAAA A1, (ZZZZZ et al.). The references are listed in a PTO-892 from the Office Action in which they are first used.
[AltContent: textbox (Prasad et al. Fig. 11, showing a screen with container results displayed.)]
PNG
media_image1.png
329
488
media_image1.png
Greyscale
Claim 1
Regarding Claim 1, Prasad et al. teach a method, performed at a computer system comprising a processor and a computer-readable medium ("systems and devices for automating and computerizing audits, tracking, and error prevention associated with container packing events and, in particular, multi-container packing events at a storage facility, yard, warehouse, or the like," paragraph [0016]), comprising:
accessing batch data describing a plurality of orders associated with a batch, wherein each order comprises a plurality of items ("systems and devices for automating and computerizing audits, tracking, and error prevention associated with container packing events and, in particular, multi-container packing events at a storage facility, yard, warehouse, or the like," paragraph [0017]);
receiving a first image from a client device, wherein the first image depicts a plurality of physical containers and a plurality of visible contained items ("capture image data and/or other sensor data of a packing area during a packing event or shift," paragraph [0018]);
applying a contained-item identification model to the first image to identify the plurality of physical containers and the plurality of visible contained items depicted in the first image, wherein the contained-item identification model is a machine-learning computer-vision model that is trained to identify containers and visible items that are contained by those containers ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024]);
generating a first set of container-item pairs, wherein each container-item pair comprises an identified container of the plurality of identified physical containers and a subset of the plurality of visible contained items, wherein the subset of visible contained items of a container-item pairs are items contained by the corresponding container that are visible in the first image ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024]);
identifying an order of the plurality of orders for each container-item pair of the set of container-item pairs based on the subset of the plurality of visible contained items associated with each container-item pair ("The container packing monitoring system may also determine if each item is placed in the correct container, a count or quantity of each item placed in each container, if an item is dropped or otherwise misplaced, and the like," paragraph [0018]);
receiving a second image from the client device, wherein the second image depicts a subset of the plurality of physical containers and a subset of the plurality of visible contained items ("In some examples, the sensors may include multiple instances of each type of sensor. For instance, camera sensors may include multiple cameras disposed at various locations," paragraph [0022] where multiple sensors teaches multiple images);
applying the contained-item identification model to the second image to identify the subset of the plurality of physical containers and the subset of the plurality of visible contained items depicted in the second image ("The data captured and generated by the sensors may be provided to the container packing monitoring system. The container packing monitoring system may combine or otherwise correlate the sensor data from the various sensors, segment the sensor data, and classify the segmented sensor data to determine objects (such as packed items) from the sensor data," paragraph [0024]);
generating a second set of container-item pairs, wherein each of the second set of container-item pairs comprises an identified container of the subset of the physical containers of the second image and the visible contained items contained by the corresponding containers ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024] where the one or more models in conjunction with the plurality of sensors teaches a second set of container-item pairs.);
identifying an order of the plurality of orders associated with the second image ("The container packing monitoring system may also determine if each item is placed in the correct container, a count or quantity of each item placed in each container, if an item is dropped or otherwise misplaced, and the like," paragraph [0018]);
identifying a container of the subset of the plurality of physical containers depicted in the second image that is not associated with the identified order of the plurality of orders ("scan the correct identifier, and/or enter an expected unit number as well as identify the correct container but may place the wrong or incorrect item (such as an adjacent item, asset, bundle of items, or package) in the container. As another example, the agent may scan an identifier of an item but fail to place the item in the cart, such as when the agent is distracted mid-pick," paragraph [0018]);
identifying that a sorting error occurred based on the identified container ("scan the correct identifier, and/or enter an expected unit number as well as identify the correct container but may place the wrong or incorrect item (such as an adjacent item, asset, bundle of items, or package) in the container. As another example, the agent may scan an identifier of an item but fail to place the item in the cart, such as when the agent is distracted mid-pick," paragraph [0018] where fail to place the item in the cart is a sorting error); and
transmitting an alert to the client device, wherein the alert causes the client device to display an instruction to a user associated with the client device to correct the sorting error ("the system may provide the agent with an alert, reminder, control signal, or notification when the container packing monitoring system detects a misplacement, a miscount, or the like associated with one or more items," paragraph [0020]).
It is recognized that the citations and evidence provided above are derived from potentially different embodiments of a single reference. Nevertheless, it would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains to employ combinations and sub-combinations of these complementary embodiments, because Prasad et al. explicitly motivates doing so at least in paragraphs [0018], [0095] and [0107] including “Although the discussion above sets forth example implementations of the described techniques, other architectures may be used to implement the described functionality and are intended to be within the scope of this disclosure. Furthermore, although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms” and otherwise motivating experimentation and optimization.
The rejection of method claim 1 above applies mutatis mutandis to the corresponding limitations of method claim 10 and apparatus claim 19 while noting that the rejection above cites to both device and method disclosures. Claims 10 and 19 are mapped below for clarity of the record and to specify any new limitations not included in claim 1.
Claim 2
Regarding claim 2, Prasad et al. teach the method of claim 1, wherein the plurality of physical containers are at least one of paper bags, plastic bags, boxes, baskets, carts, or backpacks ("As discussed herein, containers may include boxes, bins, transport handling units (THU), pallets, unit load devices (ULDs), ocean containers, airfreight units, any object that may carry or otherwise transport a product, inventory items, and the like," paragraph [0038]).
Claim 3
Regarding claim 3, Prasad et al. teach the method of claim 1, wherein the contained-item identification model comprises a convolutional neural network ("Although discussed in the context of neural networks, any type of machine learning can be used consistent with this disclosure. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS))," paragraph [0036]).
Claim 5
Regarding claim 5, Prasad et al. teach the method of claim 1, wherein the contained-item identification model is trained to output at least one of bounding boxes indicating where physical containers are located in an image, identifiers for visible contained items depicted in an image, or bounding boxes for visible contained items depicted in an image ("In some instances, the containers may not be assigned prior to packing ( e.g., the containers are identical unlabeled boxes or the like). The system may then assign a container to an order and cause the assignment to be displayed to the packing agent via the displays within the packing area, a personal electronic device assigned to the agent, or the like," paragraph [0055] where cause the assignment to be displayed teaches output bounding boxes).
Claim 8
Regarding claim 8, Prasad et al. teach the method of claim 1, wherein the batch data comprises order data for each order of the plurality of orders ("At 208, the container packing monitoring system may determine, based at least in part on the image data, a container of the one or more containers associated with each of the one or more items. For example, various containers may be placed or arrive within the packing area and may be associated with one or more orders being filled by the packing agent," paragraph [0055]).
Claim 9
Regarding claim 9, Prasad et al. teach the method of claim 1, further comprising: identifying a container associated with the identified order that is not depicted in the second image ("At 208, the container packing monitoring system may determine, based at least in part on the image data, a container of the one or more containers associated with each of the one or more items. For example, various containers may be placed or arrive within the packing area and may be associated with one or more orders being filled by the packing agent," paragraph [0055]).
Claim 10
Regarding claim 10, Prasad et al. teach a non-transitory computer-readable medium storing instructions that, when executed by a processor of a computer system("systems and devices for automating and computerizing audits, tracking, and error prevention associated with container packing events and, in particular, multi-container packing events at a storage facility, yard, warehouse, or the like," paragraph [0016]), cause the computer system to perform operations comprising:
accessing batch data describing a plurality of orders associated with a batch, wherein each order comprises a plurality of items("systems and devices for automating and computerizing audits, tracking, and error prevention associated with container packing events and, in particular, multi-container packing events at a storage facility, yard, warehouse, or the like," paragraph [0017]);
receiving a first image from a client device, wherein the first image depicts a plurality of physical containers and a plurality of visible contained items ("capture image data and/or other sensor data of a packing area during a packing event or shift," paragraph [0018]);
applying a contained-item identification model to the first image to identify the plurality of physical containers and the plurality of visible contained items depicted in the first image, wherein the contained-item identification model is a machine-learning computer-vision model that is trained to identify containers and visible items that are contained by those containers ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024]);
generating a first set of container-item pairs, wherein each container-item pair comprises an identified container of the plurality of identified physical containers and a subset of the plurality of visible contained items, wherein the subset of visible contained items of a container-item pairs are items contained by the corresponding container that are visible in the first image ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024]);
identifying an order of the plurality of orders for each container-item pair of the set of container-item pairs based on the subset of the plurality of visible contained items associated with each container-item pair ("The container packing monitoring system may also determine if each item is placed in the correct container, a count or quantity of each item placed in each container, if an item is dropped or otherwise misplaced, and the like," paragraph [0018]);
receiving a second image from the client device, wherein the second image depicts a subset of the plurality of physical containers and a subset of the plurality of visible contained items ("In some examples, the sensors may include multiple instances of each type of sensor. For instance, camera sensors may include multiple cameras disposed at various locations," paragraph [0022] where multiple sensors teaches multiple images);
applying the contained-item identification model to the second image to identify the subset of the plurality of physical containers and the subset of the plurality of visible contained items depicted in the second image ("The data captured and generated by the sensors may be provided to the container packing monitoring system. The container packing monitoring system may combine or otherwise correlate the sensor data from the various sensors, segment the sensor data, and classify the segmented sensor data to determine objects (such as packed items) from the sensor data," paragraph [0024]);
generating a second set of container-item pairs, wherein each of the second set of container-item pairs comprises an identified container of the subset of the physical containers of the second image and the visible contained items contained by the corresponding containers ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024] where the one or more models in conjunction with the plurality of sensors teaches a second set of container-item pairs.);
identifying an order of the plurality of orders associated with the second image ("The container packing monitoring system may also determine if each item is placed in the correct container, a count or quantity of each item placed in each container, if an item is dropped or otherwise misplaced, and the like," paragraph [0018]);
identifying a container of the subset of the plurality of physical containers depicted in the second image that is not associated with the identified order of the plurality of orders ("scan the correct identifier, and/or enter an expected unit number as well as identify the correct container but may place the wrong or incorrect item (such as an adjacent item, asset, bundle of items, or package) in the container. As another example, the agent may scan an identifier of an item but fail to place the item in the cart, such as when the agent is distracted mid-pick," paragraph [0018]);
identifying that a sorting error occurred based on the identified container ("scan the correct identifier, and/or enter an expected unit number as well as identify the correct container but may place the wrong or incorrect item (such as an adjacent item, asset, bundle of items, or package) in the container. As another example, the agent may scan an identifier of an item but fail to place the item in the cart, such as when the agent is distracted mid-pick," paragraph [0018] where fail to place the item in the cart is a sorting error); and
transmitting an alert to the client device, wherein the alert causes the client device to display an instruction to a user associated with the client device to correct the sorting error ("the system may provide the agent with an alert, reminder, control signal, or notification when the container packing monitoring system detects a misplacement, a miscount, or the like associated with one or more items," paragraph [0020]).
Claim 11
Regarding claim 11, Prasad et al. teach the computer-readable medium of claim 10, wherein the plurality of physical containers are at least one of paper bags, plastic bags, boxes, baskets, carts, or backpacks ("As discussed herein, containers may include boxes, bins, transport handling units (THU), pallets, unit load devices (ULDs), ocean containers, airfreight units, any object that may carry or otherwise transport a product, inventory items, and the like," paragraph [0038]).
Claim 12
Regarding claim 12, Prasad et al. teach the computer-readable medium of claim 10, wherein the contained-item identification model comprises a convolutional neural network ("Although discussed in the context of neural networks, any type of machine learning can be used consistent with this disclosure. For example, machine learning algorithms can include, but are not limited to, regression algorithms (e.g., ordinary least squares regression (OLSR), linear regression, logistic regression, stepwise regression, multivariate adaptive regression splines (MARS), locally estimated scatterplot smoothing (LOESS))," paragraph [0036]).
Claim 14
Regarding claim 14, Prasad et al. teach the computer-readable medium of claim 10, wherein the contained-item identification model is trained to output at least one of bounding boxes indicating where physical containers are located in an image, identifiers for visible contained items depicted in an image, or bounding boxes for visible contained items depicted in an image ("In some instances, the containers may not be assigned prior to packing ( e.g., the containers are identical unlabeled boxes or the like). The system may then assign a container to an order and cause the assignment to be displayed to the packing agent via the displays within the packing area, a personal electronic device assigned to the agent, or the like," paragraph [0055] where cause the assignment to be displayed teaches output bounding boxes).
Claim 17
Regarding claim 17, Prasad et al. teach the computer-readable medium of claim 10, wherein the batch data comprises order data for each order of the plurality of orders ("At 208, the container packing monitoring system may determine, based at least in part on the image data, a container of the one or more containers associated with each of the one or more items. For example, various containers may be placed or arrive within the packing area and may be associated with one or more orders being filled by the packing agent," paragraph [0055]).
Claim 18
Regarding claim 18, Prasad et al. teach the computer-readable medium of claim 10, the operations further comprising: identifying a container associated with the identified order that is not depicted in the second image ("At 208, the container packing monitoring system may determine, based at least in part on the image data, a container of the one or more containers associated with each of the one or more items. For example, various containers may be placed or arrive within the packing area and may be associated with one or more orders being filled by the packing agent," paragraph [0055]).
Claim 19
Regarding claim 19, Prasad et al. teach a system ("systems and devices for automating and computerizing audits, tracking, and error prevention associated with container packing events and, in particular, multi-container packing events at a storage facility, yard, warehouse, or the like," paragraph [0016])comprising:
a processor ("stored within the computer-readable media 712 and configured to execute on the processors 710. For example, as illustrated, the computer-readable media 712," paragraph [0094]); and
a non-transitory computer-readable medium storing instructions that ("stored within the computer-readable media 712 and configured to execute on the processors 710. For example, as illustrated, the computer-readable media 712," paragraph [0094]), when executed by a processor of a computer system, cause the computer system to perform operations comprising:
accessing batch data describing a plurality of orders associated with a batch, wherein each order comprises a plurality of items ("systems and devices for automating and computerizing audits, tracking, and error prevention associated with container packing events and, in particular, multi-container packing events at a storage facility, yard, warehouse, or the like," paragraph [0017]);
receiving a first image from a client device, wherein the first image depicts a plurality of physical containers and a plurality of visible contained items ("capture image data and/or other sensor data of a packing area during a packing event or shift," paragraph [0018]);
applying a contained-item identification model to the first image to identify the plurality of physical containers and the plurality of visible contained items depicted in the first image, wherein the contained-item identification model is a machine-learning computer-vision model that is trained to identify containers and visible items that are contained by those containers ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024]);
generating a first set of container-item pairs, wherein each container-item pair comprises an identified container of the plurality of identified physical containers and a subset of the plurality of visible contained items, wherein the subset of visible contained items of a container-item pairs are items contained by the corresponding container that are visible in the first image ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024]);
identifying an order of the plurality of orders for each container-item pair of the set of container-item pairs based on the subset of the plurality of visible contained items associated with each container-item pair ("The container packing monitoring system may also determine if each item is placed in the correct container, a count or quantity of each item placed in each container, if an item is dropped or otherwise misplaced, and the like," paragraph [0018]);
receiving a second image from the client device, wherein the second image depicts a subset of the plurality of physical containers and a subset of the plurality of visible contained items ("In some examples, the sensors may include multiple instances of each type of sensor. For instance, camera sensors may include multiple cameras disposed at various locations," paragraph [0022] where multiple sensors teaches multiple images);
applying the contained-item identification model to the second image to identify the subset of the plurality of physical containers and the subset of the plurality of visible contained items depicted in the second image ("The data captured and generated by the sensors may be provided to the container packing monitoring system. The container packing monitoring system may combine or otherwise correlate the sensor data from the various sensors, segment the sensor data, and classify the segmented sensor data to determine objects (such as packed items) from the sensor data," paragraph [0024]);
generating a second set of container-item pairs, wherein each of the second set of container-item pairs comprises an identified container of the subset of the physical containers of the second image and the visible contained items contained by the corresponding containers ("For example, the container packing monitoring system may utilize one or more machine learned models ( e.g., machine learning models, machine trained models, and/or the like) and/or networks to segment and classify the sensor data (such as the image data of the packing event). In these examples, the container packing monitoring system may then determine if the correct item and the correct number of items were placed in the correct container during the packing event," paragraph [0024] where the one or more models in conjunction with the plurality of sensors teaches a second set of container-item pairs.);
identifying an order of the plurality of orders associated with the second image ("The container packing monitoring system may also determine if each item is placed in the correct container, a count or quantity of each item placed in each container, if an item is dropped or otherwise misplaced, and the like," paragraph [0018]);
identifying a container of the subset of the plurality of physical containers depicted in the second image that is not associated with the identified order of the plurality of orders ("scan the correct identifier, and/or enter an expected unit number as well as identify the correct container but may place the wrong or incorrect item (such as an adjacent item, asset, bundle of items, or package) in the container. As another example, the agent may scan an identifier of an item but fail to place the item in the cart, such as when the agent is distracted mid-pick," paragraph [0018]);
identifying that a sorting error occurred based on the identified container ("scan the correct identifier, and/or enter an expected unit number as well as identify the correct container but may place the wrong or incorrect item (such as an adjacent item, asset, bundle of items, or package) in the container. As another example, the agent may scan an identifier of an item but fail to place the item in the cart, such as when the agent is distracted mid-pick," paragraph [0018] where fail to place the item in the cart is a sorting error); and
transmitting an alert to the client device, wherein the alert causes the client device to display an instruction to a user associated with the client device to correct the sorting error ("the system may provide the agent with an alert, reminder, control signal, or notification when the container packing monitoring system detects a misplacement, a miscount, or the like associated with one or more items," paragraph [0020]).
Claim 20
Regarding claim 20, Prasad et al. teach the system of claim 19, wherein the contained-item identification model is trained to output at least one of bounding boxes indicating where physical containers are located in an image, identifiers for visible contained items depicted in an image, or bounding boxes for visible contained items depicted in an image ("In some instances, the containers may not be assigned prior to packing ( e.g., the containers are identical unlabeled boxes or the like). The system may then assign a container to an order and cause the assignment to be displayed to the packing agent via the displays within the packing area, a personal electronic device assigned to the agent, or the like," paragraph [0055] where cause the assignment to be displayed teaches output bounding boxes).
2nd Claim Rejections - 35 USC § 103
Claims 4 and 13 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2025 0091751 A1, (Prasad et al.) in view of US Patent Publication 2025 0328859 A1, (Meunier et al.). The references are listed in a PTO-892 from the Office Action in which they are first used.
Claim 4
PNG
media_image2.png
642
373
media_image2.png
Greyscale
Regarding Claim 4, Prasad et al. teach the method of claim 1, as noted above.
Prasad et al. is not relied upon to explicitly teach all of a multi-modal large language model
[AltContent: textbox (Meunier et al. Fig. 5, showing an image of a container in front of a door for analysis.)]However, Meunier et al. teach wherein the contained-item identification model comprises a multi-modal large language model ("In one or more embodiments, the machine learning model is a multimodal Large Language Model (LLM), which acts as an image classifier and extracts spatial features of a received image at varying fields-of-view," paragraph [0065]).
Therefore, taking the teachings of Prasad et al. and Meunier et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify “System and Method for reducing Multi-container Packing Errors” as taught by Prasad et al. to use “Detecting errors in Delivered Orders using Image Analysis” as taught by Meunier et al. The suggestion/motivation for doing so would have been that, “Occasionally delivery errors occur, such as delivering an order to the wrong location, delivering the wrong order to the correct location, including an erroneous item in the order delivered, or delivering an order with a missing item.” as noted by the Meunier et al. disclosure in paragraph [0001], which also motivates combination because the combination would predictably have a lower error rate as there is a reasonable expectation that humans and robots will have errors; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 13
Regarding claim 13, Prasad et al. teach the computer-readable medium of claim 10, as noted above.
Prasad et al. is not relied upon to explicitly teach all of a multi-modal large language model.
However, Meunier et al. teach wherein the contained-item identification model comprises a multi-modal large language model ("In one or more embodiments, the machine learning model is a multimodal Large Language Model (LLM), which acts as an image classifier and extracts spatial features of a received image at varying fields-of-view," paragraph [0065]).
Prasad et al. and Meunier et al. are combined as per claim 4.
3rd Claim Rejections - 35 USC § 103
Claims 6-7 and 15-16 are rejected under 35 U.S.C. 103 as obvious over US Patent Publication 2025 0091751 A1, (Prasad et al.) in view of US Patent Publication 2022 0016779 A1, (Wang et al.). The references are listed in a PTO-892 from the Office Action in which they are first used.
Claim 6
Regarding Claim 6, Prasad et al. teach the method of claim 1, as noted above.
Prasad et al. is not relied upon to explicitly teach all of heuristics.
[AltContent: textbox (Wang et al. Fig. 2C, showing a robot packing an order into a container.)]
PNG
media_image3.png
497
657
media_image3.png
Greyscale
However, Wang et al. teach wherein identifying an order for each container-item pair of the set of container-item pairs comprises: applying a set of heuristics or a set of rules to the set of container-item pairs and the batch data ("To address the packing problem, the disclosure describes a polynomial time constructive algorithm to implement a resolution-complete search amongst feasible object placements, under robot-packable constraints, a three-dimensional positioning heuristic named Heightmap Minimization (HM) that minimizes the volume increase of the object pile from the loading direction, and a fast prioritized search scheme," paragraph [0028]).
Therefore, taking the teachings of Prasad et al. and Wang et al. as a whole, it would have been obvious to a person having ordinary skill in the art before the time of the effective filing date of the claimed invention of the instant application to modify “System and Method for reducing Multi-container Packing Errors” as taught by Prasad et al. to use “Autonomous Robot Packaging of Arbitrary Objects” as taught by Wang et al. The suggestion/motivation for doing so would have been that, “Existing automated loading systems, such as those designed for mixed palletizing of boxes, however, often cannot handle a wide range of objects reliably enough for commercial use.” as noted by the Wang et al. disclosure in paragraph [0003], which also motivates combination because the combination would predictably have a higher productivity as there is a reasonable expectation that humans and robots will be subject to errors; and/or because doing so merely combines prior art elements according to known methods to yield predictable results.
Claim 7
Regarding claim 7, Prasad et al. teach the method of claim 1, as noted above.
Prasad et al. is not relied upon to explicitly teach all of constraint satisfaction problems.
However, Wang et al. teach wherein identifying an order for each container-item pair of the set of container-item pairs comprises: applying a technique for solving a constraint satisfaction problem to the set of container-item pairs and the batch data ("The candidate placement may then be checked for constraint satisfaction. If no candidate placement is found when all options are exhausted, the method may fall back to a slower search in a five-dimensional parameter space, which includes options for the parameters in the two horizontal axes and three rotation axes," paragraph [0028]).
Prasad et al. and Wang et al. are combined as per claim 6.
Claim 15
Regarding claim 15, Prasad et al. teach the computer-readable medium of claim 10, as noted above.
Prasad et al. is not relied upon to explicitly teach all of heuristics.
However, Wang et al. teach wherein identifying an order for each container-item pair of the set of container-item pairs comprises:
applying a set of heuristics or a set of rules to the set of container-item pairs and the batch data ("To address the packing problem, the disclosure describes a polynomial time constructive algorithm to implement a resolution-complete search amongst feasible object placements, under robot-packable constraints, a three-dimensional positioning heuristic named Heightmap Minimization (HM) that minimizes the volume increase of the object pile from the loading direction, and a fast prioritized search scheme," paragraph [0028]).
Prasad et al. and Wang et al. are combined as per claim 6.
Claim 16
Regarding claim 16, Prasad et al. teach the computer-readable medium of claim 10, as noted above.
Prasad et al. is not relied upon to explicitly teach all of constraint satisfaction problem.
However, Wang et al. teach wherein identifying an order for each container-item pair of the set of container-item pairs comprises: applying a technique for solving a constraint satisfaction problem to the set of container-item pairs and the batch data ("The candidate placement may then be checked for constraint satisfaction. If no candidate placement is found when all options are exhausted, the method may fall back to a slower search in a five-dimensional parameter space, which includes options for the parameters in the two horizontal axes and three rotation axes," paragraph [0028]).
Prasad et al. and Wang et al. are combined as per claim 6.
Reference Cited
The prior art made of record and not relied upon is considered pertinent to applicant’s disclosure.
US Patent Publication 2025 0342950 A1 to Gonzalez et al. discloses detect a triggering condition by an imaging device. The system may capture, by the imaging device in response to the triggering condition, an image of a container. The system may perform, by a machine learning model, pixel quantification of the container. The system may determine, based on the pixel quantification, a status of the container.
International Patent Publication 2020 264436 A1 to Rahilly et al. discloses providing automated inventory management of medicine and healthcare items stored within bins in care facilities are disclosed. A method includes providing an interactive storage device for attaching to a bin, and outputting, via an audiovisual element, a visual representation of a local inventory of the bin, receiving a user input, determining a change to the local inventory according to the user input, updating the local inventory in a non-volatile data store according to the change, synchronizing the local inventory with one or more nodes via a communication interface, and receiving, from the one or more nodes via the communication interface, periodic updates.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to HEATH E WELLS whose telephone number is (703)756-4696. The examiner can normally be reached Monday-Friday 8:00-4:00.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Ms. Jennifer Mehmood can be reached on 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Heath E. Wells/Examiner, Art Unit 2664
Date: 1 April 2026
Prosecution Insights