DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Response to Amendment
The amendment filed on 8/13/2026 has been entered. Claims 1, 7, 15 were amended. Claims 1-20 remain pending in the application. The objections to the Specification are withdrawn due to the amendments.
Response to Arguments
Applicants’ arguments (in Remarks filed 8/13/2026) have been considered, and are not persuasive regarding the 35 USC § 101, and partially persuasive regarding the 35 USC § 102 (now converted to 35 USC § 103).
Regarding the 35 USC § 101 response and Step 2A Prong One, Applicant argues on pg 11-12, reproduced below:
PNG
media_image1.png
746
754
media_image1.png
Greyscale
In relation to the “does not identify which judicial exception the claims are alleged to recite”, the Examiner has added clarifying remarks for the groupings in the 101 rejection below.
Upon further review of the reference and in light of applicant's argument, the examiner respectfully disagrees as follows: first of all, in relation to the “unfilled template placeholder”, Applicant is respectfully reminded that typos, or written errors, do occur when writing documents. Using Applicant’s logic, if a typo, or writing error, exists, then the entire argument is wrong; this is similar to finding a typo in the instant specification and ignoring the entirety of the specification; as shown in the Non-Final Office Action where a typo was found in the specification; the entire application was not ignored for this error. Secondly, Applicant has ignored the claim limitation that is a “defining a size parameter of a pallet”, which is a mental process including observation and evaluation, and can be done mentally in the human mind because the human mind is capable of deciding to measure a size parameter of a pallet [non-limiting examples dimensions of the pallet such as width and height]. Or as the instant specifications state in para. [0041]: “the user may input the size parameter of the pallet that needs to be positioned or directly select the size parameter of the pallet from the database to construct the pallet template having predetermined pallet feature”. The user is being interpreted as having a human mind. Thirdly, Applicant cited the “adjusting a pitch…” limitation belonging to the Step 2A Prong Two analysis, not the Prong One analysis. Using the Applicant’s logic stated earlier, the entire argument should be ignored for this error in citation (in the interest of compact prosecution, Examiner has addressed this in the 101 rejection below). Fourth, “inverse transformation matrix”, as specified further in the 101 rejection below, can be done on pen and paper. The clarification provided below shows an example of calculating an inverse transformation matrix by hand.
Regarding the 35 USC § 101 response and Step 2A Prong Two, Applicant argues on pgs. 13-14, reproduced below:
PNG
media_image2.png
324
746
media_image2.png
Greyscale
PNG
media_image3.png
586
773
media_image3.png
Greyscale
Upon further review of the reference and in light of applicant's argument, the examiner respectfully disagrees as follows: first of all, regarding the citation of the specification, in response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., photographing module having a single optical axis, the resulting on-site image is subject to perspective distortion, pallets at different distances have unified size and no rotation angle in the perspective adjusted image) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Further, as MPEP 2111 states, reproduced below:
PNG
media_image4.png
552
1054
media_image4.png
Greyscale
Secondly, the BRI of the amended limitation cited are rotating a 3D object, or a pallet, so that the desired view is attained, such as having no skew or pitch, which can include the front facing side of a pallet (as Molter shows in Figure 6d).
Regarding the 35 USC § 101 response and Step 2B, Applicant argues on pgs. 14-18, where Applicant cites Example 36 that “further explains that claim 3's combination of extracting contour and character information to form feature vectors and recognizing and tracking items using those vectors and a recognition model was ‘not well-understood, routine, conventional activity in this field’ and improved upon prior inventory techniques” with the partial reproduction below (pg 17-18):
PNG
media_image5.png
622
764
media_image5.png
Greyscale
Upon further review of the reference and in light of applicant's argument, the examiner respectfully disagrees as follows: first of all, regarding the citation of the specification, in response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., a single-optical-axis image subject to perspective distortion) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Secondly, example 36 has specific mention of a recognition model and “extracting contour and character information to form feature vectors and recognizing and tracking items using those vectors”, which the instant claims do not have. The instant claims do cite a recognition model. Further, there is contour and character information used for tracking items. The BRI of the instant claims are rotating a 3D model in 3D space that has a ground plane and origin (the 101 rejection details further how software such as AutoCAD already does this) where a user may take measurements (also clarified in the 101 rejection below, AutoCAD allows a user to calculate angles and distances, as non-limiting examples). As AutoCAD software demonstrates, rotating a 3D model in 3D space that has a ground plane and origin is well-understood, routine, or conventional. Accordingly, the amended limitations, as they are currently written, do not overcome the 35 USC 101 rejection.
Regarding the 35 USC 102 rejection over Molter, the rejection has been converted into a 35 USC 103 rejection to clarify the points made.
Regarding Argument A (pg 18), Molter does not appear to specifically disclose the mechanics of rotating, but shows the results of change the camera view to see the front side of the pallet in Figure 6c, which looks similar to Figure 6 and 7 of the instant application. The 103 rejection provided below shows the mechanics of rotating a 3D object in 3D space that contains a ground plane and origin.
Regarding Argument B (pg 19), reproduced below:
PNG
media_image6.png
290
748
media_image6.png
Greyscale
Upon further review of the reference and in light of applicant's argument, the examiner respectfully disagrees as follows: first of all, regarding the citation of the specification, in response to applicant's argument that the references fail to show certain features of the invention, it is noted that the features upon which applicant relies (i.e., the projection plane coordinate is a coordinate system constructed from vectors associated with the projection datum) are not recited in the rejected claim(s). Although the claims are interpreted in light of the specification, limitations from the specification are not read into the claims. See In re Van Geuns, 988 F.2d 1181, 26 USPQ2d 1057 (Fed. Cir. 1993). Secondly, equation of a plane necessitates the existence of a normal vector (illustrative, non-limiting example here for the equation of plane: https://web.ma.utexas.edu/users/m408m/Display12-5-3.shtml ); this is part of the definition of a plane. While Molter does not explicitly disclose origin (the claimed projection plane coordinate), but does disclose pallet position (Abstract), which requires a known origin to get the position information (or projection plane coordinates), the 103 rejection prior art demonstrates that the origin is known in software such as AutoCAD; otherwise, there would be no 3D models to manipulate.
Regarding Argument C (pg 19), reproduced below:
PNG
media_image7.png
290
756
media_image7.png
Greyscale
Upon further review of the reference and in light of applicant's argument, the examiner respectfully disagrees as follows: first of all, the 103 rejection provides clarification that finding the angle for a selection in a 3D model is accomplished in AutoCAD (non-limiting example of 3D modeling software which can open point cloud data). Further, the wooden blocks refer to the wooden pallet, which is the first pallet image, which refers to the 3D point cloud data of the pallet. Secondly, Applicant appears to be referring to Molter never forming the first pallet image, as shown above, but there appears to be mention of the first pallet image never forming above. As stated above, “the wooden blocks refer to the wooden pallet, which is the first pallet image, which refers to the 3D point cloud data of the pallet”.
Regarding Argument D (pg 19-20), Examiner agrees with the inherency argument, which is why the 103 rejection was formed to provide clarification.
Accordingly, the claims, as they are currently written, do not place the instant application into an allowable state.
Claim Rejections - 35 USC § 101
Claim(s) 1-20 is/are rejected under 35 U.S.C. 101 because the claimed invention is directed to an abstract idea without significantly more. The limitations, under their broadest reasonable interpretation, cover mental process (concept performed in a human mind, including as observation, evaluation, judgment, opinion, organizing human activity and mathematical concepts and calculations). The claim(s) recite(s) a method, a system for pallet positioning. This judicial exception is not integrated into a practical application because the steps do not add meaningful limitations to be considered specifically applied to a particular technological problem to be solved .The claim(s) does/do not include additional elements that are sufficient to amount to significantly more than the judicial exception because the steps of the claimed invention can be done mentally and no additional features in the claims would preclude them from being performed as such except for the generic computer elements at high level of generality (i.e., processor, memory).
According to the USPTO guidelines, a claim is directed to non-statutory subject matter if:
STEP 1: the claim does not fall within one of the four statutory categories of invention (process, machine, manufacture or composition of matter), or
STEP 2: the claim recites a judicial exception, e.g. an abstract idea, without reciting additional elements that amount to significantly more than the judicial exception, as determined using the following analysis:
STEP 2A (PRONG 1): Does the claim recite an abstract idea, law of nature, or natural phenomenon?
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application?
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception?
Using the two-step inquiry, it is clear that claims 1, 7, 15 are directed to an abstract idea as shown below:
STEP 1: Do the claims fall within one of the statutory categories? YES. Claim(s) 1, 7, and 15 are directed to a method, i.e. process, and a system.
STEP 2A (PRONG 1): Is the claim directed to a law of nature, a natural phenomenon or an abstract idea? YES, the claims are directed toward a mental process (i.e. abstract idea).
With regard to STEP 2A (PRONG 1), the guidelines provide three groupings of subject matter that are considered abstract ideas:
Mathematical concepts – mathematical relationships, mathematical formulas or equations, mathematical calculations;
Certain methods of organizing human activity – fundamental economic principles or practices (including hedging, insurance, mitigating risk); commercial or legal interactions (including agreements in the form of contracts; legal obligations; advertising, marketing or sales activities or behaviors; business relations); managing personal behavior or relationships or interactions between people (including social activities, teaching, and following rules or instructions); and
Mental processes – concepts that are practicably performed in the human mind (including an observation, evaluation, judgment, opinion).
The method in claims 1 and 7 (and the system in claim 15) comprise a mental process that can be practicably performed in the human mind (or generic computers or components configured to perform the method) and, therefore, an abstract idea.
Regarding Claim(s) 1, 7, and 15: the method recites the steps (functions) of:
defining a size parameter of a pallet (mental process including observation and evaluation, and can be done mentally in the human mind; the human mind is capable of deciding to measure a size parameter of a pallet [non-limiting examples dimensions of the pallet such as width and height]. Or as the instant specifications state in para. [0041]: “the user may input the size parameter of the pallet that needs to be positioned or directly select the size parameter of the pallet from the database to construct the pallet template having predetermined pallet feature”. The user is being interpreted as having a human mind);
defining a pallet feature of the pallet according to the size parameter (mental process including observation and evaluation, and can be done mentally in the human mind; as the instant specifications state: “The user may define the pallet feature of the pallet by the parameters, for example, the pallet has two opening regions, etc., for constructing the pallet template 9” in para. [0042]. A human mind is capable of counting the number of openings in a pallet);
defining a projection datum in the on-site image (mental process including observation and evaluation, and can be done mentally in the human mind; a human mind can observe and evaluate if they are standing on the ground or if the ground plane exists in an on-site image. Further, the instant specifications states in para. [0043]: “the user may define the projection datum 21 in the on-site image 2”).
calculating a projection plane coordinate based on the projection datum (mental process including observation and evaluation, and can be done mentally in the human mind; the human mind is capable of calculating where the origin of a 3D coordinate system [projection plane coordinate] is based on the ground plane [projection datum]. As a non-limiting example, the user may define they are standing on the ground, and using their fingers to measure a distance, calculate the origin of the 3D coordinate system to be that finger distance from a selected corner of the pallet);
transforming the on-site image to a perspective adjusted image according to the projection plane coordinate (mental process including observation and evaluation, and can be done mentally in the human mind; the human mind is capable of imagining a 3D object, and then rotating it. As a non-limiting example, a human may be told, “Don’t think about a pink elephant”. The human will think about a pink elephant, potentially visualizing it standing on the ground. Next, the human is instructed to rotate, or transform, to view the side or the face of the pink elephant visualization in their head to create a perspective adjusted image. Further, the pink elephant or a pallet, can face the origin, or the projection plane coordinate, then can be rotated around the origin in the human’s head);
performing a first positioning procedure to obtain a first pallet image in the perspective adjusted image (mental process including observation and evaluation, and can be done mentally in the human mind; a human mind is capable of rotating a pallet image in their head so that the front face of the pallet is visualized; as cited later, Molter, Figure 4 demonstrates the mind’s visualization placed into a drawing).
calculating an inclined angle of the first pallet image (mental process including observation and evaluation, and can be done mentally in the human mind; the human mind is able is capable of calculating an angle if they know trigonometry and know the dimensions of the sides of the right triangle. For example, if the incline is the hypotenuse of a right triangle, with known height and base of 1 inch, the angle would simply be 45 degrees. Illustrative reference for common trigonometry: http://cochranmath.pbworks.com/w/page/118045008/Solving%20Trigonometric%20Equations ; see “primary values” for common degrees);
rotating the first pallet image according to the inclined angle (mental process including observation and evaluation, and can be done mentally in the human mind; as stated early, the human mind is capable of rotating a visualization, or a pallet image, in their mind);
These limitations, as drafted, is a simple process that, under their broadest reasonable interpretation, covers performance of the limitations in the mind or by a human. The Examiner notes that under MPEP 2106.04(a)(2)(III), the courts consider a mental process (thinking) that “can be performed in the human mind, or by a human using a pen and paper" to be an abstract idea. CyberSource Corp. v. Retail Decisions, Inc., 654 F.3d 1366, 1372, 99 USPQ2d 1690, 1695 (Fed. Cir. 2011). As the Federal Circuit explained, "methods which can be performed mentally, or which are the equivalent of human mental work, are unpatentable abstract ideas the ‘basic tools of scientific and technological work’ that are open to all.’" 654 F.3d at 1371, 99 USPQ2d at 1694 (citing Gottschalk v. Benson, 409 U.S. 63, 175 USPQ 673 (1972)). See also Mayo Collaborative Servs. v. Prometheus Labs. Inc., 566 U.S. 66, 71, 101 USPQ2d 1961, 1965 ("‘[M]ental processes[] and abstract intellectual concepts are not patentable, as they are the basic tools of scientific and technological work’" (quoting Benson, 409 U.S. at 67, 175 USPQ at 675)); Parker v. Flook, 437 U.S. 584, 589, 198 USPQ 193, 197 (1978) (same).
STEP 2A (PRONG 2): Does the claim recite additional elements that integrate the judicial exception into a practical application? NO, the claims do not recite additional elements that integrate the judicial exception into a practical application.
With regard to STEP 2A (prong 2), whether the claim recites additional elements that integrate the judicial exception into a practical application, the guidelines provide the following exemplary considerations that are indicative that an additional element (or combination of elements) may have integrated the judicial exception into a practical application:
an additional element reflects an improvement in the functioning of a computer, or an improvement to other technology or technical field;
an additional element that applies or uses a judicial exception to affect a particular treatment or prophylaxis for a disease or medical condition;
an additional element implements a judicial exception with, or uses a judicial exception in conjunction with, a particular machine or manufacture that is integral to the claim;
an additional element effects a transformation or reduction of a particular article to a different state or thing; and
an additional element applies or uses the judicial exception in some other meaningful way beyond generally linking the use of the judicial exception to a particular technological environment, such that the claim as a whole is more than a drafting effort designed to monopolize the exception.
While the guidelines further state that the exemplary considerations are not an exhaustive list and that there may be other examples of integrating the exception into a practical application, the guidelines also list examples in which a judicial exception has not been integrated into a practical application:
an additional element merely recites the words “apply it” (or an equivalent) with the judicial exception, or merely includes instructions to implement an abstract idea on a computer, or merely uses a computer as a tool to perform an abstract idea;
an additional element adds insignificant extra-solution activity to the judicial exception; and
an additional element does no more than generally link the use of a judicial exception to a particular technological environment or field of use.
Claim(s) 1, 7, 15 does/do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application. Claim(s) 1, 7, 15 recite(s) the further limitations of:
obtaining an on-site image (insignificant pre/post-solution extra activity of generating data; a human or machine can use an image sensor to obtain an on-site image) OR (generic computers or components configured to perform the method; a human can use a point cloud file in AutoCAD to open the point cloud file to obtain an on-site image);
obtaining a second pallet image (insignificant pre/post-solution extra activity of generating data; a human or computer could use an image sensor to obtain a second pallet image) OR (generic computers or components configured to perform the method; a human with a 3D model of a pallet can rotate the image in 3D space to obtain a second pallet image. Non-limiting examples include using AutoCAD to rotate a 3D object to obtain a second pallet image);
obtaining a position information of the pallet based on the second pallet image (generic computers or components configured to perform the method; a human with a 3D model of a pallet can use 3D model software to obtain position information. Non-limiting examples include using AutoCAD to obtain distance, such as in the following: https://help.autodesk.com/view/ACD/2026/ENU/?caas=caas/documentation/CIV3D/2014/ENU/filesACD/GUID-87EF8D09-F418-4FB4-AB62-464242AAE9EE-htm.html, titled “About Finding Distances, Angles, and Point Locations”.);
adjusting a pitch angle and a skew angle of the on-site image being projected to the projection plane coordinate based on the projection plane coordinate and a three dimensional coordinate of the projection datum to transform the on-site image to a perspective adjusted image without pitch and skew (generic computers or components configured to perform the method; a human can use 3D modeling software, such as AutoCAD, to rotate the 3D model to face the front side of a pallet where there is no pitch or skew; as Molter shows in Figure 6d);
Claim(s) 15 does/do not recite any of the exemplary considerations that are indicative of an abstract idea having been integrated into a practical application. Claim(s) 1, 7, 15 recite(s) the further limitations of:
a photographing module, obtaining an on-site image (insignificant pre/post-solution extra activity of generating data; a camera, a non-limiting example of a photographing module, to take picture of an on-site image) OR (generic computers or components configured to perform the method; a computer controlled machine with a camera taking a picture);
an on-site constructing module, electrically connected with the photographing module, configured to receive the on-site image (insignificant pre/post-solution extra activity of generating data; a computer receiving an image) OR (generic computers or components configured to perform the method; a computer receiving an image);
an input unit, configured to receive a size parameter of a pallet (generic computers or components configured to perform the method; a simple command-line interface or a GUI could allow a user to input values);
a constructing unit, electrically connected with the input unit (generic computers or components configured to perform the method; a computer that can receive numbers or input. A keyboard is an input unit electrically connected to a computer that allows a human to construct the words on screen);
a positioning computation module, electrically connected with the on-site constructing module (insignificant pre/post-solution extra activity of generating data; a non-limiting example could be a GPS module that collects position information) OR (generic computers or components configured to perform the method; a non-limiting example could be a GPS module that sends position information to a computer);
These limitations are recited at a high level of generality (i.e. as a general action or change being taken based on the results of the acquiring step) and amounts to mere post solution actions, which is a form of insignificant extra-solution activity. Further, the claims are claimed generically and are operating in their ordinary capacity such that they do not use the judicial exception in a manner that imposes a meaningful limit on the judicial exception. Accordingly, even in combination, these additional elements do not integrate the abstract idea into a practical application because they do not impose any meaningful limits on practicing the abstract idea.
STEP 2B: Does the claim recite additional elements that amount to significantly more than the judicial exception? NO, the claims do not recite additional elements that amount to significantly more than the judicial exception.
With regard to STEP 2B, whether the claims recite additional elements that provide significantly more than the recited judicial exception, the guidelines specify that the pre-guideline procedure is still in effect. Specifically, that examiners should continue to consider whether an additional element or combination of elements:
adds a specific limitation or combination of limitations that are not well-understood, routine, conventional activity in the field, which is indicative that an inventive concept may be present; or
simply appends well-understood, routine, conventional activities previously known to the industry, specified at a high level of generality, to the judicial exception, which is indicative that an inventive concept may not be present.
Claim(s) 1, 7, 15 does/do not recite any additional elements that are not well-understood, routine or conventional. The use of a computer to “defining (for claim 15), obtaining (for claim 15), and calculating (for claims 1 and 7)”, as claimed in Claim(s) 1, 7, 15 is a routine, well-understood and conventional process that is performed by computers.
Thus, since Claim(s) 1, 7, 15 is/are: (a) directed toward an abstract idea, (b) do not recite additional elements that integrate the judicial exception into a practical application, and (c) do not recite additional elements that amount to significantly more than the judicial exception, it is clear that Claim(s) 1, 7, 15 is/are not eligible subject matter under 35 U.S.C 101.
Regarding claim 8: the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. The limitation(s): performing a second positioning procedure to the first pallet image being rotated to obtain the second pallet image (mental process including observation and evaluation, and can be done mentally in the human mind; a human mind can rotate a 3D visualization of a pallet. Non-limiting examples include rotating the imaginary pallet to view the front side of the pallet where the holes are visible, or top-side) OR (generic computers or components configured to perform the method; a human can use 3D modelling software, non-limiting example of AutoCAD, to rotate the camera view around a 3D model of a pallet).
Regarding claim 2, 9, 16: the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. The limitation(s): comparing the first pallet image being rotated and the pallet feature (mental process including observation and evaluation, and can be done mentally in the human mind; a human can count the number of pallet holes, or the pallet feature, while looking at an image of the front side of the pallet).
Regarding claim 3, 10, 17: the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. The limitation(s): determining whether the second pallet image is corresponding to the pallet according to the size parameter and the pallet feature; and confirming that the second pallet image is obtained, if the second pallet image is determined to be corresponding to the pallet (mental process including observation and evaluation, and can be done mentally in the human mind; a human can do pallet identification and recognition and compare if the second pallet image is the pallet. Otherwise, forklift drivers would have trouble identifying if the pallet they were tasked to obtain was the correct one).
Regarding claim 4, 11, 18: the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. The limitation(s): comparing an edge distribution of the second pallet image and an edge distribution of the pallet feature; and aligning the edge distribution of the second pallet image and the edge distribution of the pallet feature (generic computers or components configured to perform the method; as cited later with 3DSteve, edge information can be obtained with 3D modeling software such as AutoCAD. Additionally, OpenCV has edge detection and algorithms exist to compare edge-images, illustrative and non-limiting example here: https://stackoverflow.com/questions/7715407/how-do-i-compare-two-edge-images-in-opencv . The post by Erdogan Kurtur shows a non-limiting example of Haussdorff distance. Further, aligning the edge images can be done on generic image editing software, non-limiting example of Photoshop with illustrative example in the following: https://www.photoshopessentials.com/photo-editing/how-to-auto-align-and-composite-images-in-photoshop/).
Regarding claim 12: the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. The limitation(s): positioning the first pallet image in the perspective adjusted image according to the pallet feature (mental process including observation and evaluation, and can be done mentally in the human mind; a human mind can move a pallet visualization according to the pallet feature. Non-limiting example includes rotating the visualization so that the front two holes of a pallet are visible, or the front facing side) OR (generic computers or components configured to perform the method; a human using 3D modeling software, such as AutoCAD, can rotate a 3D model of a pallet according to the pallet feature of holes is front facing, as a non-limiting example).
Regarding claim 6, 14, 20: the additional limitations do not integrate the mental process into practical application or add significantly more to the mental process. The limitation(s): obtaining an inverse transformation matrix according to the inclined angle and the position information; and transforming the position information to the three dimensional information by using the inverse transformation matrix (mental process including observation and evaluation, and can be done mentally in the human mind; using pen and paper, a human can calculate an inverse transformation matrix, a non-limiting, illustrative example is provided here: https://math.stackexchange.com/questions/152462/inverse-of-transformation-matrix , the user hardmath demonstrates this) OR (mathematical concepts, mathematical relationships, mathematical formulas or equations, mathematical calculations; as stated earlier, a human with pen and paper could calculate and obtain an inverse transformation matrix) OR (generic computers or components configured to perform the method; a human can use a matrix calculator to obtain and calculate the inverse transformation matrix, a non-limiting example includes https://matrix.reshish.com/inverse-matrix/ ).
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1-5, 7-13, 15-19 is/are rejected under 35 U.S.C. 103 as being unpatentable over Molter, in view of Rovira (“3D Understanding by Ground Plane Estimation Based on Virtual Camera Motion”, 2007) and 3DSteve (“Working with Point Clouds in AutoCAD”, 2016).
Regarding claims 1, 7, 15, Molter teaches A pallet positioning method (Molter, pg 297, column 2, ¶1, reproduced below:
PNG
media_image8.png
490
1052
media_image8.png
Greyscale
. “Detecting and localizing pallets” is being interpreted as “pallet positioning”), for obtaining three dimensional information (Molter, Abstract, “Our algorithm is based on finding surfaces in
the point cloud”, which is being interpreted to involve 3D information) of a pallet (Molter, Abstract, “The detection and localization result in the pallets position and orientation in relation to the camera coordinate system”. Pallet localization is being interpreted as involving 3D information) by perspective adjustment (Molter, pg 301, Figure 6d, which shows a perspective adjusted image without pitch and skew. Which looks similar to instant application Fig. 7, which is also similar to Molter’s Figure 4 condition 2 and 4, the front view of a pallet on the ground. The BRI for these limitations is moving the camera so that the front of the pallet, or 3d object, is in view), comprising:
defining a size parameter of the pallet (Molter, pg 300, Figure 4, reproduced below:
PNG
media_image9.png
730
704
media_image9.png
Greyscale
. Examiner recommends viewing the color version. “range of valid values” is being interpreted as “size parameter of a pallet” that are defined);
defining a pallet feature of the pallet according to the size parameter (Molter, see Figure 4 image above, “geometrical conditions” are being interpreted as “pallet feature” according to the size parameter that are defined);
obtaining an on-site image (Molter, see pg 297, column 2, ¶1 image above, “3D
camera with time-of-flight principle is used” is being interpreted as being used to obtain “an on-site image”);
defining a projection datum in the on-site image (Molter, pg 299, column 1, ¶1, reproduced below:
PNG
media_image10.png
344
718
media_image10.png
Greyscale
. “For plane detection, the well-known RANSAC algorithm is used” is being interpreted to involve “projection datum”);
adjusting a pitch angle and a skew angle (Molter, pg 301, Figure 6, going from 6c to 6d shows the camera view is adjusting a pitch and skew angle to view the front face of the pallet) of the on-site image (Molter, pg 301, Figure 6c, which shows a non-limiting example of the on-site image)…
…transforming the on-site image to a perspective adjusted image without pitch and skew (Molter, pg 301, Figure 6d, which shows a perspective adjusted image without pitch and skew. Which looks similar to instant application Fig. 7, which is also similar to Molter’s Figure 4 condition 2 and 4, the front view of a pallet on the ground. The BRI for these limitations is moving the camera so that the front of the pallet, or 3d object, is in view);
obtaining a second pallet image (Molter, pg 301, Figure 6d, which is being interpreted as the second pallet image);
obtaining a position information of the pallet (Molter, see pg 297, column 2, ¶1 image above, “localizing pallets” are being interpreted to involve position information) based on the second pallet image (Molter, pg 301, Figure 6d, which is being interpreted to have position information, otherwise the pallet would not be visible in the image); and
obtaining a three dimensional information of the pallet according to the inclined angle and the position information (Molter, see pg 297, column 2, ¶1 image above, “localizing pallets” are being interpreted to involve obtaining 3D information of the pallet according to the inclined angle, interpreted from yaw angle, and position information, interpreted from the geometrical conditions from Figure 4).
However, Molter does not appear to specifically teach calculating a projection plane coordinate (interpreted as a non-limiting example of 3D Cartesian coordinate origin). Although, Molter does teach determining ground plane equation (Molter, pg 299, column 1, ¶1-3) and pallets position (Molter, Abstract), which shows a 3D coordinate system with origin exists.
Pertaining to the same field of endeavor, Rovira teaches
calculating a projection plane coordinate (Rovira, pg 3, Section 2.1, paragraph after equation 1: “origin of the coordinate system” is a non-limiting example of projection plane coordinate) based on the projection datum (Rovira, pg 3, Section 2, ¶1: “we estimate the ground plane”. Which is being interpreted as a non-limiting example of projection datum, or the ground plane estimation);
…being projected to the projection plane coordinate (Rovira, pg 3, Section 2.1, paragraph after equation 1: “origin of the coordinate system” is a non-limiting example of projection plane coordinate) based on the projection plane coordinate (Rovira, pg 3, Section 2.1, paragraph after equation 1: “origin of the coordinate system” is a non-limiting example of projection plane coordinate) and a three dimensional coordinate of the projection datum to (Rovira, pg 3, Section 2, ¶1: “we estimate the ground plane”. Which is being interpreted as a non-limiting example of projection datum, or the ground plane estimation)…
Molter and Rovira are considered to be analogous art because they are directed to 3D understanding. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for pallet position and angle determination with ground plane determination (as taught by Molter) to include calculating a projection plane coordinate because the combination provides to obtain useful object information such as real position, orientation (Rovira, Abstract).
However, Molter and Rovira does not appear to explicitly teach 3D model object rotation. Although, Molter does teach camera view change that appears to be a camera rotation in Figure 6. And Rovira teach Virtual Camera Motion for rotation axis determination (Rovira, Abstract).
Pertaining to the same field of endeavor, 3DSteve teaches
rotating the first pallet image (3DSteve, pg 3, 1:20 transcript: “So, I can rotate this…”) according to the inclined angle (3DSteve, pg 5, 2:30 transcript: “And I’ll rotate this to 90 degrees”. Which is a non-limiting example of the incline angle. Pages 6-8 sample images (2:21, 2:23, 2:27), show the user picking out two points for the reference angle, then the user rotates the image in 2:30 on page 9);
Molter, Rovira, and 3DSteve are considered to be analogous art because they are directed to 3D understanding. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for pallet position and angle determination with ground plane determination, modified to calculate projection plane (as taught by Molter and Rovira) to include 3D model object rotation (as taught by 3DSteve) because the combination provides to obtain useful object information such as real position, orientation (Rovira, Abstract). Further, 3D model object rotation is common practice. A non-limiting example exists at the 3D Transformer website (link: https://www.3dtransformer.com/), which shows rotating 3D objects in 3D space would be known to PHOSITA. As a non-limiting example, the computer animated series Reboot (1994-2001) shows 3D models in 3D space. If the animators could not rotate or get dimensions of the 3D models, then the animation and modeling would not be possible. Additionally, PHOSITA would know obtaining 3D information from 3D models (such as angles) is possible, as a non-limiting example, AutoCAD allows for that (link: https://help.autodesk.com/view/ACD/2026/ENU/?caas=caas/documentation/CIV3D/2014/ENU/filesACD/GUID-87EF8D09-F418-4FB4-AB62-464242AAE9EE-htm.html ) in their document titled “About Finding Distances, Angles, and Point Locations”.
Regarding claim 8, Molter teaches The pallet positioning method according to claim 7, further comprising:
performing a second positioning procedure to the first pallet image being rotated to obtain the second pallet image (Molter, Figure 6, reproduced below:
PNG
media_image11.png
530
608
media_image11.png
Greyscale
. Examiner recommends viewing the full color version. The visualization shows the view change from 6c to 6d. One with ordinary skill in the art would know this view change requires at least a rotation, which involves a second positioning procedure. 6c is interpreted as an example first pallet image. 6d is interpreted as an example of a second pallet image).
Regarding claim 9, Molter teaches The pallet positioning method according to claim 8, wherein the second positioning procedure comprises:
comparing the first pallet image being rotated (Molter, see nearest image below, the pallet detected in the point cloud is being interpreted to involve the first pallet image. One with ordinary skill in the art would know that point cloud images are easily rotated) and the pallet feature (Molter, pg 299, Section C, ¶1, reproduced below:
PNG
media_image12.png
378
608
media_image12.png
Greyscale
. The pallet characterizations are being interpreted as involving the pallet feature. The known dimensions are compared with the pallet features and the wooden blocks detected).
Regarding claim 10, Molter teaches The pallet positioning method according to claim 7, wherein the obtaining of the second pallet image comprises:
determining whether the second pallet image (Molter, Figure 6d, which is being interpreted as the second pallet image) is corresponding to the pallet according to the size parameter (Molter, pg 300, column 1, lines 1-2: “The Euclidean distance between all remaining centroids is calculated”. “Euclidean distance” is being interpreted to involve an example “size parameter”) and the pallet feature (Molter, pg 299, column 2, last paragraph: “The idea here is to find three centroids, which represents the front side surface of the three wooden blocks.” The 3 centroids that represents the 3 wooden blocks is being interpreted to involve the pallet feature. All together, these determine if the object detected corresponds to the pallet); and
confirming that the second pallet image is obtained (Molter, pg 300, Figure 4 text, “All four geometrical conditions for pallet detection.” “Geometrical conditions for pallet detection” is being interpreted as confirming that the second pallet image is obtained), if the second pallet image is determined to be corresponding to the pallet (Molter, pg 300, Figure 4 text, “All four geometrical conditions for pallet detection.” “Geometrical conditions for pallet detection” is being interpreted as using geometrical pallet detection to determine if the object detected corresponds to the pallet).
Regarding claim 11, Molter teaches The pallet positioning method according to claim 10, wherein the obtaining of the second pallet image further comprises:
comparing an edge distribution (Molter, see nearest image below, “Each cluster represents one found surface”, which is being interpreted as an “edge distribution”) of the second pallet image (Molter, Figure 6d, which shows the second pallet image with centroids shown) and an edge distribution of the pallet feature (Molter, pg 299, column 2, ¶1, reproduced below:
PNG
media_image13.png
266
608
media_image13.png
Greyscale
. “Surface normal” is being interpreted to involve “the pallet feature” as it is used to determine if the point cloud clusters are the surfaces, or edge distribution, or the 3 wooden blocks. Calculating the angle phi between is being interpreted as comparing); and
aligning the edge distribution of the second pallet image (Molter, see nearest image above, the point cloud cluster of the found surface is being interpreted as the edge distribution of the second pallet image) and the edge distribution of the pallet feature (Molter, see nearest image above, surface normal is being interpreted as involving the edge distribution of the pallet. “Perpendicular” surface normal are being interpreted as an example aligning).
Regarding claim 12, Molter teaches The pallet positioning method according to claim 7, wherein the first positioning procedure comprises:
positioning the first pallet image in the perspective adjusted image according to the pallet feature (Molter, Figure 6, reproduced below:
PNG
media_image11.png
530
608
media_image11.png
Greyscale
. Examiner recommends viewing the full color version. 6c is being interpreted as the first pallet image. The visualized centroids are the red line, which is being interpreted as involving a pallet feature. The position of the pallet view is being interpreted to involve a perspective adjusted image).
Regarding claim 13, Molter teaches The pallet positioning method according to claim 7, wherein the position information (Molter, see pg 297, column 2, ¶1 image above, “localizing pallets” are being interpreted to involve position information) is a two dimensional information (Molter, pg 297, column 1, ¶1: “localization as ‘what is the spatial orientation in relation to the sensor?’.” Which is being interpreted to involve two dimensional information.).
Claims 2 and 16 is rejected using the same rationale as applied to claim 9 discussed above.
Claims 3 and 17 is rejected using the same rationale as applied to claim 10 discussed above.
Claims 4 and 18 is rejected using the same rationale as applied to claim 11 discussed above.
Claims 5 and 19 is rejected using the same rationale as applied to claim 13 discussed above.
Claim(s) 6, 14, 20 is/are rejected under 35 U.S.C. 103 as being unpatentable over Molter, as modified by Rovira and 3DSteve, in view of Holeva (US 2013/0101230 A1, 2012).
Regarding claim 14, Molter teaches The pallet positioning method according to claim 7, wherein the obtaining of the three dimensional information of the pallet according to the inclined angle and the position information comprises:
However, Molter does not appear to explicitly teach inverse transformation matrix. Although, one with ordinary skill in the art would know inverse transformation matrix is used in robotic navigation with an image sensor, or used in changing the frame of reference for different coordinate systems.
Pertaining to the same field of endeavor, Holeva teaches
obtaining an inverse transformation matrix (Holeva, [0708], reproduced below:
PNG
media_image14.png
354
822
media_image14.png
Greyscale
. “wherein the noted inverse matrix”, which is being interpreted as involving an inverse transformation matrix) according to the inclined angle (Holeva, see [0708] image above, the theta is being interpreted as involving the inclined angle) and the position information (Holeva, [0707], reproduced below:
PNG
media_image15.png
386
890
media_image15.png
Greyscale
. The X, Y, and Z is being interpreted as position information); and
transforming the position information to the three dimensional information (Holeva, see [0707] image above, the X, Y, Z values are being interpreted to be transformed into 3D information) by using the inverse transformation matrix (Holeva, see [0708] image above, which is the inverse transformation matrix used).
Molter, Rovira, 3DSteve, and Holeva are considered to be analogous art because they are directed to 3D understanding. It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the method and system for pallet position and angle determination with ground plane determination, modified to calculate projection plane, further modified to include 3D model object rotation (as taught by Molter, Rovira, 3DSteve) to include inverse transformation matrix (as taught by Holeva) because the combination provides an improvement to forklift operation (Holeva, [0004]).
Claims 6 and 20 is rejected using the same rationale as applied to claim 13 discussed above.
Conclusion
THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a).
A shortened statutory period for reply to this final action is set to expire THREE MONTHS from the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the mailing date of this final action and the advisory action is not mailed until after the end of the THREE-MONTH shortened statutory period, then the shortened statutory period will expire on the date the advisory action is mailed, and any nonprovisional extension fee (37 CFR 1.17(a)) pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, however, will the statutory period for reply expire later than SIX MONTHS from the mailing date of this final action.
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Xiao et al (“Pallet recognition and localization using an RGB-D camera”, 2017) discloses a pallet positioning (interpreted from localization) device that defines pallet features and parameters (see Figure 5) with similarity matching (interpreted from templated matching), first and second pallet image (segmentation and Figures 7d to 7f), and inclined angle (example could be yaw angle).
Knitt et al (“Estimating the Pose of a Euro Pallet with an RGB Camera based on Synthetic Training Data”, 2022) discloses pallet pose estimation (which uses pallet feature and size parameters) with inclined angle (pg 6, column 1, first paragraph).
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JOHNNY B DUONG whose telephone number is (571)272-1358. The examiner can normally be reached Monday - Thursday 10a-9p (ET).
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Matthew Bella can be reached at (571)272-7778. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/J.B.D./Examiner, Art Unit 2667
/MATTHEW C BELLA/Supervisory Patent Examiner, Art Unit 2667