DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The instant application claims priority to and benefit of Japanese Application No. JP2023-169601, filed on 09/29/2023.
Applicant’s response to the last Office Action dated 06/11/2026, as well as amendment to claims, filed on 08/19/2026, have been entered and made of record.
Status of Claims
Claims 1 and 5-7 are currently pending. Claims 2-4 are canceled.
The amendments of Claims 1 and 5-6 are accepted and entered.
Response to Arguments
Applicant remarks filed on 08/19/2026 have been carefully considered. In light of Applicant amending the claims in order to recite the structures responsible to perform the recited tasks in the claims, Examiner agrees with Applicant’s remarks that the claims of the instant application are no longer interpreted under 35 U.S.C. 112(f). In the course of search and consideration for the amended claims, Examiner discovered additional prior art that tracks to the amended claims. Therefore, new analyses are presented with respect to the amended claims. This action is made non-final.
Claim Objections
Claim 5 is objected to for the following informalities: an extra comma before the end period. Appropriate correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
6. Claims 1 and 5-6 are rejected under 35 U.S.C. 103 as being unpatentable over Onoda et. al. (US 20220375206) in view of Zhao et. al. (“Recognition and Location Algorithm for Pallets in Warehouses Using RGB-D Sensor” with publishing date of October 13, 2022) in further view of Souder et. al. (US 20180322362),
Consider Claim 1, the closest prior art with the same field of endeavor, Onoda, teaches “A pallet recognition device that recognizes a position and a state of a pallet, the pallet recognition device comprising: a memory configured to store pieces of reference data acquired in advance, the pieces of reference data corresponding to a plurality of types of pallets;” (Onoda; Abstract; “A pallet deviation detection section previously stores position/shape data of the pallets and performs comparison between the stored position/shape data corresponding to the identified type of the target pallet and the position/shape data of the target pallet.” (emphasis added)) “a camera configured to determine a type of the pallet;” (Onoda; [0004]; “The cameras 3 and 4 take an image of the pattern 01 of the pallet 05. The position determination means determines a position of the pallet 05 from the taken image of the pattern, and the comparison means compares the taken image of the pattern and a reference pattern with each other to identify a type of the pallet 05.” (emphasis added)) “a laser sensor configured to acquire detection data of the pallet;” (Onoda; [0002]; “The laser range finder 18 performs scanning while applying laser beams to an opening end face 19a of a pallet 19 disposed at a load pickup position, to obtain image data of a line of an edge of the hole 20.”) “at least one processor configured to: (Onoda; Abstract; “ A pallet deviation detection section previously stores position/shape data of the pallets and performs comparison between the stored position/shape data corresponding to the identified type of the target pallet and the position/shape data of the target pallet.”) “(Onoda; Abstract; “A pallet deviation detection section previously stores position/shape data of the pallets and performs comparison between the stored position/shape data corresponding to the identified type of the target pallet and the position/shape data of the target pallet.” (emphasis added)) “evaluate the estimation values of the position and the state of the pallet calculated; determine the position and the state of the pallet based on a result of evaluation; wherein the at least one processor is further configured to: “calculate the estimation values of the position and the state of the pallet by matching the detection data and the reference data, and calculate a degree of matching between the detection data and the reference data; evaluate the estimation values of the position and the state of the pallet by determining whether the degree of matching between the detection data and the reference data is equal to or greater than a threshold determined in advance; and determine the estimation values of the position and the state of the pallet calculated as the position and the state of the pallet when it is determined that the degree of matching between the detection data and the reference data is equal to or greater than the threshold, and estimate the position and the state of the pallet based on the detection data when it is determined that the degree of matching between the detection data and the reference data is less than the threshold.”.
Onoda does not explicitly disclose the limitations: “evaluate the estimation values of the position and the state of the pallet calculated; determine the position and the state of the pallet based on a result of evaluation; wherein the at least one processor is further configured to: calculate the estimation values of the position and the state of the pallet by matching the detection data and the reference data, and calculate a degree of matching between the detection data and the reference data; evaluate the estimation values of the position and the state of the pallet by determining whether the degree of matching between the detection data and the reference data is equal to or greater than a threshold determined in advance; and determine the estimation values of the position and the state of the pallet calculated as the position and the state of the pallet when it is determined that the degree of matching between the detection data and the reference data is equal to or greater than the threshold”. However, in analogous field of endeavor, Zhao teaches “evaluate the estimation values of the position and the state of the pallet calculated; determine the position and the state of the pallet based on a result of evaluation;” (Zhao; 2.6 Template Matching; “The position of the pallet was set to be determined by matching the template and category matrix with the sliding window method [48,49]…. The template matrix matches the category matrix to calculate the matching degree…. A higher matching score corresponds to a higher probability of being a pallet. If the matching score is higher than MiniScore, the pallet position is determined.” (emphasis added)) “wherein the at least one processor is further configured to: “calculate the estimation values of the position and the state of the pallet by matching the detection data and the reference data, and calculate a degree of matching between the detection data and the reference data;” (Zhao; 2.6 Template Matching; “The position of the pallet was set to be determined by matching the template and category matrix…. The template matrix matches the category matrix to calculate the matching degree….”) “evaluate the estimation values of the position and the state of the pallet by determining whether the degree of matching between the detection data and the reference data is equal to or greater than a threshold determined in advance; and” (Zhao; 2.2. Algorithm Flow; “ A template is created based on the target information. To accelerate the matching process, both the category matrix and template are compressed. The labeled template is matched to the category matrix, and the match score of pixels is calculated.”) “determine the estimation values of the position and the state of the pallet calculated as the position and the state of the pallet when it is determined that the degree of matching between the detection data and the reference data is equal to or greater than the threshold,” (Zhao; 2.6 Template Matching; “A higher matching score corresponds to a higher probability of being a pallet. If the matching score is higher than MiniScore, the pallet position is determined. MiniScore is determined with a threshold, as shown in Equation (12).”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Onoda with the teachings of Zhao to determine a pallet type based on a calculated matching degree. One of ordinary skill in the art would be motivated to combine Onoda and Zhao to further analyze the detected pallet type using a calculated matching score to output a more accurate identification of a target pallet. Accordingly, the combination of Onoda and Zhao discloses the above described limitations of Claim 1.
The combination of Onoda and Zhao does not explicitly teach “and estimate the position and the state of the pallet based on the detection data when it is determined that the degree of matching between the detection data and the reference data is less than the threshold.” However, in an analogous field of endeavor, Souder teaches “estimate the position and the state of the pallet based on the detection data when it is determined that the degree of matching between the detection data and the reference data is less than the threshold.” (Souder; [0056]; “In some embodiments, a threshold matching score may be established for which a match may be identified. For example, the vector comparison engine 612 may only identify a match or matches having a matching score greater than or equal to 80%. In some embodiments, a threshold matching score may be established for which a match may be identified…. If the matching score is below the threshold (e.g., under 80%), the vector comparison engine 612 may characterize the pallet being analyzed as a pallet that does not already exist in the database 603 (e.g., a new pallet or preexisting pallet that has not previously been analyzed and/or entered into the database 603, or a pallet that has changed visually). Thus, the vector comparison engine 612 may create a new entry in the database 603 for the pallet being analyzed. The entry may include a pallet identifier (e.g., a unique identification code, name or number corresponding to the pallet being analyzed) and the set of vectors generated by the vector generation engine 611.”; Examiner notes Souder teaches “The vector generation engine 611 may be configured to, in conjunction with the processor 601, receive the image, identified visual features, and spatial orientations from the spatial orientation identification engine 610 and/or the feature identification engine 609.” (emphasis added) (Souder; [0053])). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Onoda and Zhao with the teachings of Souder to further determine the pallet type and position of a pallet using the detection data if the matching threshold is not met. One of ordinary skill in the art would be motivated to combine Onoda, Zhao and Souder to detect and store the new pallet type for “…the next time the pallet is seen in the supply chain, it may be visually identified using the methods described herein” (Souder; [0056]). Accordingly, the combination of Onoda, Zhao, and Souder discloses the invention of Claim 1.
Consider Claim 5, the combination of Onoda, Zhao, and Souder teaches “The pallet recognition device according to at least one processor Is further configured to extract pieces of the detection data matching the reference data from the detection data (Onoda; Abstract; “ A pallet position/shape obtaining section obtains position/shape data of the target pallet from a distance measuring device for measuring a distance to the target pallet. A pallet deviation detection section previously stores position/shape data of the pallets…” (emphasis added)) “estimate the position and the state of the pallet based on the pieces of detection data matching the reference data when it is determined that the degree of matching between the detection data and the reference data is less than the threshold.” (Souder; [0056]; “In some embodiments, a threshold matching score may be established for which a match may be identified…. If the matching score is below the threshold (e.g., under 80%), the vector comparison engine 612 may characterize the pallet being analyzed as a pallet that does not already exist in the database 603 (e.g., a new pallet or preexisting pallet that has not previously been analyzed and/or entered into the database 603, or a pallet that has changed visually). Thus, the vector comparison engine 612 may create a new entry in the database 603 for the pallet being analyzed. The entry may include a pallet identifier (e.g., a unique identification code, name or number corresponding to the pallet being analyzed) and the set of vectors generated by the vector generation engine 611.”). The proposed combination as well as the motivation for combining the Onoda, Zhao and Souder references presented in the rejection of claim 1, apply to claim 5 and are incorporated herein by reference. Thus, the method recited in claim 5 is met by Onoda, Zhao and Souder.
Consider Claim 6, the combination of Onoda, Zhao, and Souder teaches “The pallet recognition device according to claim 1, wherein the (Onoda; [0004]; “The cameras 3 and 4 take an image of the pattern 01 of the pallet 05.”) “and determines the type of the pallet based on image data from the camera,” (Onoda; [0013]; “an image obtaining section that obtains a taken image from an imaging device for taking an image of a portion in front of the unmanned forklift…the taken image of the target pallet, which is obtained by the image obtaining section;…”) “the laser sensor irradiates the pallet with a laser and receives reflected light of the laser to detect a distance to the pallet and” (Onoda; [0009]; “Meanwhile, for example, in a case where a plurality a type of pallets are detected for position deviation by only a distance detector such as a laser scanner, actually measured shape data obtained by the distance detector needs to be compared with all types of stored position/shape data of a plurality of types of pallets, in order to identify a type of the corresponding pallet.”) “acquire detection point cloud data,” (Zhao; 2.7. Pose Parameters of Pallet Estimation; “In the camera coordinate system {C}, the calculation formula of the pallet pose parameter is shown as Equations (13) and (14), respectively. 𝑃𝐶 is calculated by averaging the corresponding point cloud data in the detection grid of the center pallet feet.”) “and the memory stores pieces of reference point cloud data corresponding to the plurality of types of the pallets.” (Onoda; [0102]; “L0 in the schematic plan view illustrated in FIG. 8A represents, for example, a line segment based on the stored points data of the height including the insertion opening Q of the pallet P corresponding to the target pallet P1 in FIG. 2 or FIG. 3, regarding pallets on the floor N.”). The proposed combination as well as the motivation for combining the Onoda, Zhao, and Souder references presented in the rejection of claim 1, apply to claim 6 and are incorporated herein by reference. Thus, the method recited in claim 6 is met by Onoda, Zhao, and Souder.
7. Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Onoda et. al. (US 20220375206) in view of Zhao et. al. (“Recognition and Location Algorithm for Pallets in Warehouses Using RGB-D Sensor” with publishing date of October 13, 2022) in further view of Souder et. al. (US 20180322362), and in still further view of Zaccaria et. al. (“A Comparison of Deep Learning Models for Pallet Detection in Industrial Warehouses” with publishing date of November 26, 2020).
Consider Claim 7, the combination of Onoda, Zhao, and Souder does not explicitly disclose “The pallet recognition device according to claim 1, wherein one of the pieces of the reference data of the plurality of types of pallets is reference data for a state in which the pallet is partially covered by another object.”. However, in an analogous field of endeavor, Zaccaria teaches “The pallet recognition device according to claim 1, wherein one of the pieces of the reference data of the plurality of types of pallets is reference data for a state in which the pallet is partially covered by another object.” (Zaccaria; Section I: Introduction; “Therefore, in this work we also present a new dataset of RGB images that was acquired from an industrial setting (warehouse). Images of the dataset contain pallets in various configurations, i.e. images may contain multiple pallets at different heights, either on the ground or on racks. Moreover, pallets may have an arbitrary orientation. Furthermore, pallets can be partially covered by a transparent plastic wrap film.”).
Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Onoda, Zhao, and Souder with the teachings of Zaccaria to include images of partially cover pallets in the reference dataset for pallet type detection. One of ordinary skill in the art would be motivated to combine Onoda, Zhao, Souder, and Zaccaria to address one of the main drawbacks of computer vision “such as detection of objects that are not available in standard datasets (e.g. pallets)” (Zaccaria, Section I: Introduction). Accordingly, the combination of Onoda, Zhao, Souder, and Zaccaria discloses the invention of Claim 7.
Conclusion
8. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Annie Pham whose telephone number is (571)272-1673. The examiner can be normally be reached Mon-Fri 9:00a – 5:00p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANNIE H PHAM/Examiner, Art Unit 2662
/Siamak Harandi/Primary Examiner, Art Unit 2662
Prosecution Insights