DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Priority
The instant application claims priority to and benefit of Japan Application No. 2023-169601, filed on 09/29/2023. Thus, the effective filing date of Claims 1-7 are 09/29/2023.
Information Disclosure Statement
The information disclosure statements (“IDS”) filed on 09/23/2024 and 12/11/2025 was reviewed and the listed references were noted.
Drawings
The 7 page drawings have been considered and placed on record in the file.
Status of Claims
Claims 1-7 are currently pending.
Claim Rejections - 35 USC § 112(f)
The following is a quotation of 35 U.S.C. 112(d):
(d) REFERENCE IN DEPENDENT FORMS.—Subject to subsection (e), a claim in dependent form shall contain a reference to a claim previously set forth and then specify a further limitation of the subject matter claimed. A claim in dependent form shall be construed to incorporate by reference all the limitations of the claim to which it refers.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitations are: "a detection unit configured to…”, “a selection unit configured to …”, “a data acquisition unit configured to …”, “an evaluation unit configured to …”, and “a position-state determination unit configured to …”, in Claims 1-7.
Because these claim limitations are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it is being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this limitation interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
Determining the scope and contents of the prior art.
Ascertaining the differences between the prior art and the claims at issue.
Resolving the level of ordinary skill in the pertinent art.
Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
9. Claims 1-3 and 6 are rejected under 35 U.S.C. 103 as being unpatentable over Onoda et. al. (US 20220375206) in view of Zhao et. al. (“Recognition and Location Algorithm for Pallets in Warehouses Using RGB-D Sensor” with publishing date of October 13, 2022) in further view of Shogo et. al. (JP 2020109030).
Consider Claim 1, Onoda teaches “A pallet recognition device that recognizes a position and a state of a pallet, the pallet recognition device comprising: (Onoda; Abstract; “A pallet deviation detection section previously stores position/shape data of the pallets and performs comparison between the stored position/shape data corresponding to the identified type of the target pallet and the position/shape data of the target pallet.” (emphasis added)) “a detection unit configured to determine a type of the pallet;” (Onoda; Abstract; “A pallet type identification section has a learning model for combinations of images of a plurality of types of pallets and types of the pallets”) “a selection unit configured to select reference data corresponding to the type of the pallet detected by the detection unit from the pieces of reference data of the plurality of types of pallets stored in the memory;” (Onoda; Abstract; “A pallet deviation detection section previously stores position/shape data of the pallets and performs comparison between the stored position/shape data corresponding to the identified type of the target pallet and the position/shape data of the target pallet.” (emphasis added)) “a data acquisition unit configured to acquire detection data of the pallet; and” (Onoda; Abstract; “ A pallet deviation detection section previously stores position/shape data of the pallets and performs comparison between the stored position/shape data corresponding to the identified type of the target pallet and the position/shape data of the target pallet.” (emphasis added))
Onoda does not explicitly disclose “a matching processor configured to calculate estimation values of a position and a state of the pallet by matching the detection data acquired by the data acquisition unit and the reference data selected by the selection unit.”. However, in an analogous field of endeavor, Zhao teaches “a matching processor configured to calculate estimation values of a position and a state of the pallet by matching the detection data acquired by the data acquisition unit and the reference data selected by the selection unit.” (Zhao; 2.6 Template Matching; “The position of the pallet was set to be determined by matching the template and category matrix with the sliding window method [48,49]…. The template matrix matches the category matrix to calculate the matching degree…. A higher matching score corresponds to a higher probability of being a pallet. If the matching score is higher than MiniScore, the pallet position is determined. MiniScore is determined with a threshold, as shown in Equation (12).” (emphasis added)). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine Onoda with the teachings of Zhao to determine a pallet type based on a calculated matching degree. One of ordinary skill in the art would be motivated to combine Onoda and Zhao to further analysis the detected pallet type using a calculated matching score to output a more accurate identification of a target pallet. Accordingly, the combination of Onoda and Zhao discloses the above described limitations of Claim 1.
The combination of Onoda and Zhao does not explicitly disclose “a memory configured to store”. However, in an analogous field of endeavor, Shogo teaches “a memory configured to store” (Shogo; [0055]; “As shown in FIG. 6, the computer 5 includes a CPU 6, a main memory 7, a storage 8, and an interface 9.”). Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Onoda and Zhao with the teachings of Shogo to further implement the pallet detection system on a determination device used on a warehouse truck or device. One of ordinary skill in the art would be motivated to combine Onoda, Zhao, and Shogo to assist a forklift operator or driver to detect the type of pallet of the target to more safety lift and transport a pallet (Shogo, [0002-0005]). Accordingly, the combination of Onoda, Zhao, and Shogo discloses the invention of Claim 1.
Consider Claim 2, the combination of Onoda, Zhao, and Shogo teaches “The pallet recognition device according to claim 1, further comprising: an evaluation unit configured to evaluate the estimation values of the position and the state of the pallet calculated by the matching processor; and” (Onoda; Abstract; “A pallet deviation detection section previously stores position/shape data of the pallets and performs comparison between the stored position/shape data corresponding to the identified type of the target pallet and the position/shape data of the target pallet.” (emphasis added)) “a position-state determination unit configured to determine the position and the state of the pallet based on a result of evaluation by the evaluation unit.” (Onoda; [0112]; “The pallet shape determination section 16 performs comparison between shape data of the target pallet P1, which is obtained by the pallet position/shape obtaining section 13, and previously stored shape data of the same type of pallet as the target pallet P1. In a case where the difference therebetween is greater than a predetermined threshold value, the obtained shape data of the target pallet P1 is determined as being erroneous.”; Examiner notes Onoda teaches performing a comparison between the stored pallet information and the detected pallet position and shape to determine deviations and errors within the target pallet. As a result of the comparison, Onoda further teaches comparing the error to a predetermined threshold to decide whether to proceed with the initial position/shape detected for further image processing (Onoda, [0021-0023])).
Consider Claim 3, the combination of Onoda, Zhao, and Shogo teaches “The pallet recognition device according to claim 2, wherein the matching processor calculates the estimation values of the position and the state of the pallet by matching the detection data and the reference data, and” (Onoda; [0101]; “The pallet deviation detection section 14 performs comparison between the stored position/shape data corresponding to the type of the target pallet P1, which is identified by the pallet type identification section 12, i.e., the position/shape data of the pallet P corresponding to the type of the target pallet P1 in the previously stored pallets P data as described above, and the position/shape data of the target pallet P1, which is obtained by the pallet position/shape obtaining section 13, so as to detect deviation of a position and an orientation of the target pallet P1 from a normal position.”) “calculates a degree of matching between the detection data and the reference data, and” (Zhao; 2.2. Algorithm Flow; “ A template is created based on the target information. To accelerate the matching process, both the category matrix and template are compressed. The labeled template is matched to the category matrix, and the match score of pixels is calculated.”) “the evaluation unit evaluates the estimation values of the position and the state of the pallet by determining whether the degree of matching between the detection data and the reference data is equal to or greater than a threshold determined in advance.” (Zhao; 2.6 Template Matching; “The position of the pallet was set to be determined by matching the template and category matrix with the sliding window method [48,49]…. The template matrix matches the category matrix to calculate the matching degree…. A higher matching score corresponds to a higher probability of being a pallet. If the matching score is higher than MiniScore, the pallet position is determined. MiniScore is determined with a threshold, as shown in Equation (12).”). The proposed combination as well as the motivation for combining the Onoda, Zhao, and Shogo references presented in the rejection of claim 1, apply to claim 3 and are incorporated herein by reference. Thus, the method recited in claim 3 is met by Onoda, Zhao, and Shogo.
Consider Claim 6, the combination of Onoda, Zhao, and Shogo teaches “The pallet recognition device according to claim 1, wherein the detection unit has a camera that captures an image of the pallet, and determines the type of the pallet based on image data from the camera,” (Onoda; [0013]; “an image obtaining section that obtains a taken image from an imaging device for taking an image of a portion in front of the unmanned forklift…the taken image of the target pallet, which is obtained by the image obtaining section;…”; Examiner notes Onoda teaches an imaging device “implemented by, for example, a monocular camera…” (Onoda; [0077])) “the data acquisition unit irradiates the pallet with a laser and receives reflected light of the laser to detect a distance to the pallet and” (Onoda; [0009]; “Meanwhile, for example, in a case where a plurality of types of pallets are detected for position deviation by only a distance detector such as a laser scanner, actually measured shape data obtained by the distance detector needs to be compared with all types of stored position/shape data of a plurality of types of pallets, in order to identify a type of the corresponding pallet.”) “acquire detection point cloud data,” (Zhao; 2.7. Pose Parameters of Pallet Estimation; “In the camera coordinate system {C}, the calculation formula of the pallet pose parameter is shown as Equations (13) and (14), respectively. 𝑃𝐶 is calculated by averaging the corresponding point cloud data in the detection grid of the center pallet feet.”) “and the memory stores pieces of reference point cloud data corresponding to the plurality of types of the pallets.” (Onoda; [0102]; “L0 in the schematic plan view illustrated in FIG. 8A represents, for example, a line segment based on the stored points data of the height including the insertion opening Q of the pallet P corresponding to the target pallet P1 in FIG. 2 or FIG. 3, regarding pallets on the floor N.”). The proposed combination as well as the motivation for combining the Onoda, Zhao, and Shogo references presented in the rejection of claim 1, apply to claim 6 and are incorporated herein by reference. Thus, the method recited in claim 6 is met by Onoda, Zhao, and Shogo.
10. Claim 7 is rejected under 35 U.S.C. 103 as being unpatentable over Onoda et. al. (US 20220375206) in view of Zhao et. al. (“Recognition and Location Algorithm for Pallets in Warehouses Using RGB-D Sensor” with publishing date of October 13, 2022) in further view of Shogo et. al. (JP 2020109030), and in still further view of Zaccaria et. al. (“A Comparison of Deep Learning Models for Pallet Detection in Industrial Warehouses” with publishing date of November 26, 2020).
Consider Claim 7, the combination of Onoda, Zhao, and Shogo does not explicitly disclose “The pallet recognition device according to claim 1, wherein one of the pieces of the reference data of the plurality of types of pallets is reference data for a state in which the pallet is partially covered by another object.”. However, in an analogous field of endeavor, Zaccaria teaches “The pallet recognition device according to claim 1, wherein one of the pieces of the reference data of the plurality of types of pallets is reference data for a state in which the pallet is partially covered by another object.” (Zaccaria; Section I: Introduction; “Therefore, in this work we also present a new dataset of RGB images that was acquired from an industrial setting (warehouse). Images of the dataset contain pallets in various configurations, i.e. images may contain multiple pallets at different heights, either on the ground or on racks. Moreover, pallets may have an arbitrary orientation. Furthermore, pallets can be partially covered by a transparent plastic wrap film.”).
Accordingly, before the effective filing date of the instant application, it would have been obvious to one of ordinary skill in the art to combine the teachings of Onoda, Zhao, and Shogo with the teachings of Zaccaria to include images of partially cover pallets in the reference dataset for pallet type detection. One of ordinary skill in the art would be motivated to combine Onoda, Zhao, Shogo, and Zaccaria to address one of the main drawbacks of computer vision “such as detection of objects that are not available in standard datasets (e.g. pallets)” (Zaccaria, Section I: Introduction). Accordingly, the combination of Onoda, Zhao, Shogo, and Zaccaria discloses the invention of Claim 7.
Allowable Subject Matter
11. Claims 4-5 are objected to as being dependent upon a rejected base claim, but could be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims. The following is a statement of reasons for the indication of allowable subject matter:
Consider Claim 4, the combination of Onoda, Zhao, and Shogo teaches “The pallet recognition device according to claim 3, wherein the position-state determination unit determines the estimation values of the position and the state of the pallet calculated by the matching processor as the position and the state of the pallet when the evaluation unit determines that the degree of matching between the detection data and the reference data is equal to or greater than the threshold, and” (Zhao; 2.6 Template Matching; “The position of the pallet was set to be determined by matching the template and category matrix with the sliding window method [48,49]…. The template matrix matches the category matrix to calculate the matching degree…. A higher matching score corresponds to a higher probability of being a pallet. If the matching score is higher than MiniScore, the pallet position is determined. MiniScore is determined with a threshold, as shown in Equation (12).”) “estimates the position and the state of the pallet based on the detection data acquired by the data acquisition unit” (Onoda; [0077]; “The imaging device S1 is implemented by, for example, a monocular camera, and takes an image of a portion forward of the unmanned forklift 1 at a predetermined stop position at which the unmanned forklift 1 is stopped. The distance measuring device S2 is implemented by, for example, two-dimensional light detection and ranging (2D-LiDAR)) or a time of flight (TOF) camera, and performs distance measurement for a target pallet P1 (for example, FIG. 2) on a floor N at a predetermined stop position at which the unmanned forklift 1 is stopped. In a case where the distance measuring device S2 is 2D-LiDAR, pulse-like laser light is applied while the direction is changed to the horizontal direction, and scattered light that is reflected and returned is detected, to measure, for example, a distance to and a direction of a target object based on a time until return of the light reflected by the object.”) None of the cited prior art references, alone or in combination, provides a motivation to teach the ordered combination of the limitations recited in Claim 4 with the limitations of claims it depends from.
Consider Claim 5, the combination of Onoda, Zhao, and Shogo teaches “The pallet recognition device according to claim 4, wherein the position-state determination unit extracts pieces of the detection data matching the reference data from the detection data acquired by the data acquisition unit, and” (Onoda; Abstract; “ A pallet position/shape obtaining section obtains position/shape data of the target pallet from a distance measuring device for measuring a distance to the target pallet. A pallet deviation detection section previously stores position/shape data of the pallets…” (emphasis added)) “estimates the position and the state of the pallet based on the pieces of detection data matching the reference data” (Onoda; Abstract; “ A pallet position/shape obtaining section obtains position/shape data of the target pallet from a distance measuring device for measuring a distance to the target pallet.”) “
Conclusion
12. Any inquiry concerning this communication or earlier communications from the examiner should be directed to Annie Pham whose telephone number is (571)272-1673. The examiner can be normally be reached Mon-Fri 9:00a – 5:00p.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Amandeep Saini can be reached on (571)272-3382. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For
additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ANNIE H PHAM/Examiner, Art Unit 2662
/Siamak Harandi/Primary Examiner, Art Unit 2662
Prosecution Insights