DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
Applicant’s amendments to the claims and the abstract, filed on March 02, 2026, have been entered and made of record.
Currently pending Claim(s) 1-2, 4, and 6-8
Independent Claim(s) 1 and 7
Amended Claim(s) 1-2, 4, and 6-8
Canceled Claim(s) 3, 5, and 9
Response to Arguments
This office action is responsive to Applicant’s Arguments/Remarks Made in an Amendment received on March 02, 2026.
In view of amendments filed on March 02, 2026, the Applicant has amended the Abstract to remove legal phraseology. Specifically, some instances of the word “said” have been removed, but one instance still remains. Please see the objection in the Specification section below.
Regarding the claims, the Applicant has amended claim 8 in response to the Examiner’s previous objections. The phrase “non-transitory computer program” has been amended to “a storage device for a computer program.” Although the “storage device” in the newly amended claim 8 is not specified to be non-transitory and could be interpreted as a signal, claim 8 is not rejected under 35 U.S.C. § 101 (similarly to the rejection to claim 9 in the Non-Final Rejection, dated December 12, 2025), since it is dependent upon claim 1. Thus, there are no further objections to the claims, but the Examiner does recommend specifying that the storage device in non-transitory.
Furthermore, the Applicant has amended claims 1 and 7 to now include content from the canceled claims, and the Applicant provides arguments against the Examiner’s previous rejections and objections to the claims and drawings, respectively.
In view of Applicant Arguments/Remarks filed March 02, 2026, with respect to the drawings, the Applicant argued (pages 1-2) that the Examiner’s objection (from the previous office action, Non-Final Rejection, dated December 12, 2025) to Fig. 5 is incorrect, and that Fig. 5 does not contain “unlabeled boxes.” Rather, the boxes in Fig. 5 are representative elements identified by reference numerals. The Examiner finds this argument to be persuasive and withdraws the objection to the drawings. The boxes are representative elements and not mere boxes of a flowchart. Thus, labels are not required for understanding of the drawings, and the drawings are accepted.
With respect to the claims, the Applicant argued (Remarks page 3) that independent claim 1, which now incorporates subject matter of claims 3 and 5, is allowable over Kubota (US 12,148,178 B2) and Konolige (US 9,102,055 B1), because Kubota and Konolige do not teach determining the total cost or score regarding distances between predefined object features.
Upon further review of the arguments and the prior art, the Examiner finds this argument to be persuasive. When performing template matching for an object in the image, Kubota teaches selecting the template with the highest degree of matching to the object [Col. 4, lines 44-49]. Furthermore, Kubota teaches considering the object’s relationship to other objects, since the height of the stack of objects determines the apparent size of the object (since it is closer to the camera positioned above the stack) undergoing template matching [Col. 4, line 50 – Col. 5, line 21]. However, as stated in the previous office action, Kubota fails to teach minimizing a total cost or maximizing a total score regarding distance between predefined object model features and corresponding object features identified in the image to be closest to said predefined object model features of respective object model instance, and the Examiner instead relied on Konolige [Col. 15, lines 4-32] for teaching this limitation.
Konolige teaches taking predetermined information into account when selecting 3D virtual models to represent detected boxes in the virtual 3D space. A 3D model of the stack of boxes may be created to help the depalletization robot plan and track progress for loading/unloading boxes from a pallet [Col. 10, lines 31-34], and the robot may recognize edges of boxes while considering predetermined constraints, such as the expected range of object dimensions [Col. 15, lines 26-41]. However, the quotes from [Cols. 15-16] provided in the Examiner’s previous rejections failed to show developing a cost function or score for refining hypothetical transforms using predetermined information.
Other sections of Konolige, [Cols. 17-18], not previously quoted by the Examiner teach generating box hypotheses for boxes within a stack on a pallet, and the hypotheses are refined using scores and the predetermined range of box dimensions as constraints. Additionally, other patents by Konolige, such as (US 9,327,406 B1), teach nearly the same invention as taught in (US 9,102,055 B1) while more clearly discussing updating a score to refine the box hypothesis using predetermined information. Therefore, the Examiner presents new rejections using the patent (US 9,327,406 B1) as prior art. Accordingly, this action is made non-final.
Specification
Applicant is reminded of the proper language and format for an abstract of the disclosure.
It should avoid using phrases which can be implied, such as, “The disclosure concerns,” “The disclosure defined by this invention,” “The disclosure describes,” etc. In addition, the form and legal phraseology often used in patent claims, such as “means” and “said,” should be avoided.
The abstract of the disclosure is objected to because it includes legal phraseology. The third sentence of the abstract reads, “The matching starts from the object model instances having hypothetical transforms, respectively, in the image for matching with the imaged object instances, and wherein said matching is based on transforming respective object model instance to more accurately match with respective imaged object instance in the image.”
Legal phraseology should be removed to overcome this rejection. A corrected abstract of the disclosure is required and must be presented on a separate sheet, apart from any other text. See MPEP § 608.01(b).
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: device(s) in claim 7.
Because this claim limitation is being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it is being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
The device(s) are interpreted as generic computers with processing circuitry, memory, storage, and I/O circuitry as shown in Fig. 4 and [Page 23, line 15 – Page 24, line 15] of the Specification (dated January 18, 2024).
If applicant does not intend to have this limitation interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation to avoid it being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 102
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless –
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale, or otherwise available to the public before the effective filing date of the claimed invention.
(a)(2) the claimed invention was described in a patent issued under section 151, or in an application for patent published or deemed published under section 122(b), in which the patent or application, as the case may be, names another inventor and was effectively filed before the effective filing date of the claimed invention.
Claims 1 and 6-8 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by Hinterstroisser et al. (US 9,327,406 B1), hereafter Hinterstroisser.
Regarding claim 1, Hinterstroisser teaches a method, performed by one or more devices (Figs. 1(a-b) and 2(a-c) show a robotic device for interacting with stacked objects. [Col. 2, lines 8-11] “The system may include at least one processor, and data storage comprising instructions executable by the at least one processor to cause the system to perform operations.”),
for matching of object model instances with imaged object instances in an image, said object model instances being instances of an object model of an object, wherein said matching starts from the object model instances having hypothetical transforms, respectively, in the image for matching with the imaged object instances (Hinterstroisser teaches methods for recognizing boxes or other objects (for example, a coffee mug is recognized by the system at [Col. 16, line 60]) in a stack to be loaded/unloaded to a pallet. [Col. 2, lines 10-14] “The operations may include receiving one or more images of a physical environment, where the one or more images include one or more objects.” [Col. 4, lines 24-30] “The system may then employ one or more object recognition algorithms to determine various object hypotheses and segment the objects based on detected features. The system may determine a type (or types) of surface feature that is predicted to be contained on a portion of one or more surfaces of a single object.” Furthermore, see Cols. 13-14 discussing different methods for generating a ‘box hypothesis.’), and
wherein said matching is based on transforming respective object model instance to more accurately match with respective imaged object instance in the image (Hinterstroisser teaches creating a ‘box hypothesis’ (object model instance) and matching it to each surface of each detected object. [Col. 4, lines 42-53] “…where a box hypothesis represents estimated boundaries of distinct objects in the physical environment, Such as estimated boundaries of a single box face of a box… The term ‘box hypothesis’ may also refer to hypotheses associated with other types of physical objects as well.” [Col. 5, lines 4-13] “In some scenarios, the system may determine the segmentation based on the identified line segments in the adjusted image of the box hypothesis. Further, the system may determine a template of a box face or other object surface and compare the template with the adjusted image of the box hypothesis. Still further, the system may determine a confidence score associated with the segmentation based on a trained feature classifier configured to distinguish between true positives and false positives of determined segmentations.”), wherein the method comprises:
obtaining predetermined information regarding how the imaged object instances relate to each other in the image in addition to what said object model as such discloses about respective imaged object instance in the image (Hinterstroisser teaches obtaining predetermined information. For example, the system may learn or obtain the range of dimensions that boxes are expected to have and use that information to refine box hypothesis when recognizing multiple boxes in a stack of boxes. [Col. 14, lines 52-58] “The computing device may determine whether the refined box hypothesis violates predetermined ‘box dimension constraints’ and/or the perpendicularity of the box dimension constraints. Each box may have three different side lengths (e.g., a height, width, and depth), and the ‘box dimension constraints’ refer to lengths of two of these sides that are included in a given box hypothesis.”); and
performing said matching using said obtained predetermined information, wherein the matching comprises transformations of the object model instances that take into account said predetermined information regarding how the imaged object instances relate to each other in the image and minimizing a total cost or maximizing a total score ([Col. 13, line 64 – Col. 15, line 59] teaches the refinement and global reasoning modules which refine the box hypothesis for finer matching. This process involves determining a confidence value and/or similarity score for the matching of a box to a box hypothesis. Furthermore, predetermined information, such as the expected dimensions of a box, may be obtained and used to further constrain the matching refinement. Hinterstroisser teaches that the refined box hypothesis should “substantially match” the predefined dimension constraints. Therefore, the predetermined information further constrains the confidence score, which is a score determined for refining the box hypothesis. [Col. 14, lines 58-65] “As such, the computing device may determine that a box hypothesis does not violate the box dimension constraints if (i) the lengths of the two sides fall within a predetermined range of lengths associated with the predetermined box dimension constraints (e.g., predetermined based on a multitude of known, different boxes) and/or (ii) the two side lengths substantially match one or more predetermined box dimension constraints.”),
said total cost or total score comprising a first cost or score, provided by a first function, regarding distance between predefined object model features and corresponding object features identified in the image to be closest to said predefined object model features of respective object model instance (Hinterstroisser teaches determining a confidence value and/or a similarity score for refining the matching of a box hypothesis to a box. [Col. 13, line 64 – Col. 15, line 59] teach many different methods for generating a box hypothesis, refining the box hypothesis, and determining the confidence and/or similarity score of the matching.),
wherein said total cost or total score further comprises one or more second costs or scores provided by one or more second functions regarding deviation from how the imaged object instances relate to each other in the image according to said predetermined information (When refining a box hypothesis, the refinement module considers the predetermined information about the box’s dimensions, and a high confidence score is dependent upon the box hypothesis and the predetermined information being “substantially similar” [Col. 14, lines 58-65]. This requires a score to determine if the box hypothesis is substantially similar to the constraints of the predetermined information.).
Regarding claim 6, Hinterstroisser teaches the method as claimed in claim 1, wherein said hypothetical transforms of the object model instances are result from a preceding matching step that has been performed and resulted in said hypothetical transforms of the object model instances (Througout [Cols. 13-14], Hinterstroisser teaches different possible implementations for the “box hypothesis module,” which generates a box hypothesis. Then, throughout [Cols. 14-15], Hinterstroisser teaches the verification and refinement modules which refine the box hypothesis using predetermined information; see [Col. 14, lines 52-58]. Furthermore, the data is processed through modules in successive order, so the processing done by the refinement and verification modules occurs after the preceding step of generating the box hypotheses using the box hypothesis generation module. [Col. 13, lines 3-5] “In some implementations, such a method may be divided into different modules which may be processed in successive order.”).
Regarding claim 7, Hinterstroisser teaches a device (Figs. 1(a-b) and 2(a-c) show a robotic device for interacting with stacked objects. [Col. 2, lines 8-11] “The system may include at least one processor, and data storage comprising instructions executable by the at least one processor to cause the system to perform operations.”)
for matching of object model instances with imaged object instances in an image, said object model instances being instances of an object model of an object, wherein said matching starts from the object model instances having hypothetical transforms, respectively, in the image for matching with the imaged object instances (Hinterstroisser teaches methods for recognizing boxes or other objects (for example, a coffee mug is recognized by the system at [Col. 16, line 60]) in a stack to be loaded/unloaded to a pallet. [Col. 2, lines 10-14] “The operations may include receiving one or more images of a physical environment, where the one or more images include one or more objects.” [Col. 4, lines 24-30] “The system may then employ one or more object recognition algorithms to determine various object hypotheses and segment the objects based on detected features. The system may determine a type (or types) of surface feature that is predicted to be contained on a portion of one or more surfaces of a single object.” Furthermore, see Cols. 13-14 discussing different methods for generating a ‘box hypothesis.’), and
wherein said matching is based on transforming respective object model instance to more accurately match with respective object instance in the image (Hinterstroisser teaches creating a ‘box hypothesis’ (object model instance) and matching it to each surface of each detected object. [Col. 4, lines 42-53] “…where a box hypothesis represents estimated boundaries of distinct objects in the physical environment, Such as estimated boundaries of a single box face of a box… The term ‘box hypothesis’ may also refer to hypotheses associated with other types of physical objects as well.” [Col. 5, lines 4-13] “In some scenarios, the system may determine the segmentation based on the identified line segments in the adjusted image of the box hypothesis. Further, the system may determine a template of a box face or other object surface and compare the template with the adjusted image of the box hypothesis. Still further, the system may determine a confidence score associated with the segmentation based on a trained feature classifier configured to distinguish between true positives and false positives of determined segmentations.”), wherein said one or more devices are configured to:
obtain predetermined information regarding how the imaged object instances relate to each other in the image in addition to what said object model as such discloses about respective imaged object instance in the image (Hinterstroisser teaches obtaining predetermined information. For example, the system may learn or obtain the range of dimensions that boxes are expected to have and use that information to refine box hypothesis when recognizing multiple boxes in a stack of boxes. [Col. 14, lines 52-58] “The computing device may determine whether the refined box hypothesis violates predetermined ‘box dimension constraints’ and/or the perpendicularity of the box dimension constraints. Each box may have three different side lengths (e.g., a height, width, and depth), and the ‘box dimension constraints’ refer to lengths of two of these sides that are included in a given box hypothesis.”); and
perform said matching using said obtained predetermined information, wherein the matching comprises transformations of the object model instances that take into account said predetermined information regarding how the imaged object instances relate to each other in the image and minimizing a total cost or maximizing a total score ([Col. 13, line 64 – Col. 15, line 59] teaches the refinement and global reasoning modules which refine the box hypothesis for finer matching. This process involves determining a confidence value and/or similarity score for the matching of a box to a box hypothesis. Furthermore, predetermined information, such as the expected dimensions of a box, may be obtained and used to further constrain the matching refinement. Hinterstroisser teaches that the refined box hypothesis should “substantially match” the predefined dimension constraints. Therefore, the predetermined information further constrains the confidence score, which is a score determined for refining the box hypothesis. [Col. 14, lines 58-65] “As such, the computing device may determine that a box hypothesis does not violate the box dimension constraints if (i) the lengths of the two sides fall within a predetermined range of lengths associated with the predetermined box dimension constraints (e.g., predetermined based on a multitude of known, different boxes) and/or (ii) the two side lengths substantially match one or more predetermined box dimension constraints.”),
said total cost or total score comprising a first cost or score, provided by a first function, regarding distance between predefined object model features and corresponding object features identified in the image to be closest to said predefined object model features of respective object model instance (Hinterstroisser teaches determining a confidence value and/or a similarity score for refining the matching of a box hypothesis to a box. [Col. 13, line 64 – Col. 15, line 59] teach many different methods for generating a box hypothesis, refining the box hypothesis, and determining the confidence and/or similarity score of the matching.),
wherein said total cost or total score further comprises one or more second costs or scores provided by one or more second functions regarding deviation from how the imaged object instances relate to each other in the image according to said predetermined information (When refining a box hypothesis, the refinement module considers the predetermined information about the box’s dimensions, and a high confidence score is dependent upon the box hypothesis and the predetermined information being “substantially similar” [Col. 14, lines 58-65]. This requires a score to determine if the box hypothesis is substantially similar to the constraints of the predetermined information.).
Regarding claim 8, Hinterstroisser teaches a storage device for a computer program comprising instructions that when executed by one or more processors (Figs. 1(a-b) and 2(a-c) show a robotic device for interacting with stacked objects. [Col. 2, lines 8-11] “The system may include at least one processor, and data storage comprising instructions executable by the at least one processor to cause the system to perform operations.”) causes one or more devices to perform the method according to claim 1 (See the rejection to claim 1.).
Allowable Subject Matter
Claims 2 and 4 are objected to as being dependent upon a rejected base claim, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
Regarding claim 2, the closest prior art of record, Hinterstroisser (US 9,327,406 B1) teaches the method as claimed in claim 1, wherein said predetermined information is about one or more of the following: that the imaged object instances have the same one or more dimensions in the image ([Col. 14, lines 53-65] “The computing device may determine whether the refined box hypothesis violates predetermined ‘box dimension constraints’ and/or the perpendicularity of the box dimension constraints. Each box may have three different side lengths (e.g., a height, width, and depth), and the ‘box dimension constraints’ refer to lengths of two of these sides that are included in a given box hypothesis. As such, the computing device may determine that a box hypothesis does not violate the box dimension constraints if (i) the lengths of the two sides fall within a predetermined range of lengths associated with the predetermined box dimension constraints (e.g., predetermined based on a multitude of known, different boxes) and/or (ii) the two side lengths substantially match one or more predetermined box dimension constraints.”),
that the imaged object instances have the same shape in the image (Hinterstroisser teaches utilizing the predetermined information that the objects are simple boxes, and the box hypothesis that are recognized as distinct objects (not a flat plane of a box) above a threshold or score are filtered out. [Col. 12, lines 51-59] “Such a method may involve operations such as: preprocessing ortho graphic images, extracting corner features and line features (e.g., line segments) from the preprocessed images, building initial reasonable hypotheses from combinations of line segments, combinations of corners, single planes, and corner contours, refining the initial hypotheses, filtering out hypotheses with threshold high probability of being associated to a distinct object/surface.”),
that the imaged object instances do not overlap each other in the image ([Col. 12, lines 57-64] “…performing global reasoning based on assumptions and predictions that a given set of physical objects may not overlap.”).
However, Hinterstroisser fails to teach wherein said predetermined information is about one or more of the following: that the imaged object instances have the same rotation in the image, that the imaged object instances have a predefined rotation in the image in relation to one or more closest neighboring imaged object instances in the image, and that the imaged object instances in one or more directions should have no gap between them in the image.
In [Cols. 15-18], Hinterstroisser describes global reasoning which is performed by the robotic devices. This reasoning considers the positional relationships between boxes in a stack for refining and filtering-out box hypotheses. For example, in [Col. 12, lines 57-64] and [Col. 15, lines 18-54], Hinterstroisser teaches considering that the boxes are in a stack and should not overlap with one another when refining hypotheses and adjusting the confidence score.
Thus, Hinterstroisser teaches refining hypotheses and updating a confidence score by considering the predetermined information regarding the expected dimensions of boxes and that the boxes should not overlap. However, Hinterstroisser does not specifically teach obtaining predetermined information about rotations of objects or gaps between objects and using this predetermined information for refining a box hypothesis with a cost function or score.
Other prior art teaches similar inventions to Hinterstroisser that also fail to teach these specific limitations. For example, Konolige (US 9,102,055 B1) teaches obtaining predetermined information regarding the dimensions of boxes and refining the initial box hypotheses using the predetermined information [Cols. 17-18], but similarly to Hinterstroisser, Konolige does not teach obtaining predetermined information regarding the relative rotations of the boxes or the gaps in between. Similarly, Watts (US 9,802,317 B1) teaches nearly an identical invention to Konolige and fails to teach obtaining predetermined information regarding the relative rotations of the boxes or the gaps in between.
Other cited sources teach well-known template matching techniques similar to the claimed invention. Xu (US20140072217A1; cited in the IDS dated January 18, 2024) teaches recognizing objects by matching an object instance/template to an object in the image. The methods involve rotating the template to perform template matching with objects in an image at any orientation or rotation [Figs. 10-11]; however, these methods do not consider using a second cost function which considers predetermined positional relationships between the objects. Similarly, Gao (A New Approach of Template Matching and Localization Based on the Guidance of Feature Points. 2018 IEEE ICIA. pp. 548-553; cited in the IDS dated April 10, 2025), teaches methods for matching of an object template to an object in an image using a cost function. This method is similar to the claimed invention and can perform template matching on objects at different orientations. However, Gao also teaches performing template matching for objects without considering predetermined positional information regarding multiple objects in the image.
Regarding claim 4, Hinterstroisser teaches the method as claimed in claim 1, wherein said predetermined information that is taken into account by said transformations comprises that the imaged object instances have the same one or more dimensions in the image ([Col. 14, lines 53-65] “The computing device may determine whether the refined box hypothesis violates predetermined ‘box dimension constraints’ and/or the perpendicularity of the box dimension constraints. Each box may have three different side lengths (e.g., a height, width, and depth), and the ‘box dimension constraints’ refer to lengths of two of these sides that are included in a given box hypothesis. As such, the computing device may determine that a box hypothesis does not violate the box dimension constraints if (i) the lengths of the two sides fall within a predetermined range of lengths associated with the predetermined box dimension constraints (e.g., predetermined based on a multitude of known, different boxes) and/or (ii) the two side lengths substantially match one or more predetermined box dimension constraints.”) and/or
have the same shape in the image (Hinterstroisser teaches utilizing the predetermined information that the objects are simple boxes, and the box hypothesis that are recognized as distinct objects (not a flat plane of a box) above a threshold or score are filtered out. [Col. 12, lines 51-59] “Such a method may involve operations such as: preprocessing ortho graphic images, extracting corner features and line features (e.g., line segments) from the preprocessed images, building initial reasonable hypotheses from combinations of line segments, combinations of corners, single planes, and corner contours, refining the initial hypotheses, filtering out hypotheses with threshold high probability of being associated to a distinct object/surface.”).
However, Hinterstroisser fails to teach wherein said predetermined information that is taken into account by said transformations comprises that the imaged object instances have the same rotation in the image and/or have a predefined rotation in the image in relation to one or more closest neighboring imaged object instances in the image.
As discussed above regarding claim 2, Hinterstroisser and the other cited prior art fail to teach wherein the predetermined information is about the predefined rotation between objects in the image. See the discussion regarding claim 2 above.
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Watts et al. (US 9,802,317 B1) teaches a robotic device for picking boxes from a stack on a pallet. The robot can recognize boxes by first generating a box hypothesis for a box in the stack and then refining the box hypothesis using predetermined information about the expected dimensions or position in the environment.
Konolige et al. (US 9,630,320 B1) teaches a robotic device for picking boxes from a stack on a pallet. The robot recognizes boxes and creates a 3D virtual environment representing the work environment and the boxes. When recognizing a box to interact with, the robot generates a box hypothesis and refines it using predetermined information about the expected dimensions of a box.
Sigal et al. (US 7,436,980 B2) teaches systems and methods for object detection and tracking in videos. The method involves recognizing and tracking an object by comparing it to a predetermined spatio-temporal model which shows the relationships of components which make-up the object. The probability/confidence score that the object is in the video is determined upon recognizing each component which is part of the object.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ERIC JAMES SHOEMAKER whose telephone number is (571)272-6605. The examiner can normally be reached Monday through Friday from 8am to 5pm ET.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner' s supervisor, JENNIFER MEHMOOD, can be reached at (571)272-2976. The fax phone number for the organization where this application or proceeding is assigned is (571)273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/Eric Shoemaker/
Patent Examiner
/JENNIFER MEHMOOD/Supervisory Patent Examiner, Art Unit 2664
Prosecution Insights