DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submissions, filed on June 04th, 2026, have been entered.
Information Disclosure Statement(s)
The Information Disclosure Statement filed on April 10th, 2026 has been acknowledged and considered.
Status of Claims
Claims 1-2, 4-7, 9-10, 12-13, 15-18, 20 and 22 are pending, claims 1, 12 and 22 have been amended. Claims 1-2, 4-7, 9-10, 12-13, 15-18, 20 and 22 remains rejected.
Response to Argument(s)
101 rejection:
The examiner finds the amendment to the independent claims 1, 12 and 22 have overcome the previously stated 101 rejection, Applicants’ arguments regarding the 101 rejection found to persuasive. The amendment has integrated the additional elements of the claims into a practical application that can perform control action that are sample time independent to move actuators accordingly with respect to the load, moreover, the Applicants interprets that the candidate actions and control actions are sample time independent which is independent of rate at which the system samples its inputs or states, which is beneficial, as it makes the system well suited to variable latency control, improving the solutions.
rejections:
In view of the Amendments to independent claims 1, 12 and 22 the previously applied prior art rejections are withdrawn. Applicants’ arguments are rendered moot in view of the new grounds of rejection set forth below.
The examiner finds the Applicants’ argument regarding the limitation:
wherein the one or more action candidates are sample time independent;
…sample time independent action candidates…;
to be persuasive.
However, some argument regarding the feature of the claims of “an apparatus” to be non-persuasive, see Reply below.
Regarding the amended feature of “…sample time independent control action…” which narrows down the scope of the claim, and introduce new features to the claims, hence, required updated search and new grounds of rejection, see 103 rejections below.
However, In pages 13-19 of the remarks, the Applicants argue that the proposed prior arts Lee in view of Shum for rejection of the claims 1-6, 8-10, 12-17, 19-20 and 22, and Lee in view of Shum and Diao for the rejection of claims 7 and 18, wherein these proposed prior arts, alone or in combination, does not each or suggest the features of the currently amended independent claims 1, 12 and 22 such as:
“an apparatus…;
In support of the above arguments, the Applicants state that the independent claims such as claim 1, recites an (singular) apparatus, that is one singular apparatus perform the steps of the claims, which is different to the proposed arts in a sense that the art such as Lee teaches different methods for providing 3D information on the movement of a spreader, hence indicating different possible apparatus between conventional and proposed methods, multiple tests were performed to compare performance of the methods hence, cannot reflect one singular apparatus.
Examiner’s reply:
The examiner respectfully finds the Applicants’ arguments to be non-persuasive and incommensurate with the scope of the claim. the Applicants are respectfully reminded that the claims are construed based on BRI (broadest reasonable interpretation) in light of the specification. Therefore, the examiner recognizes the BRI scope of the claim involving the apparatus as part of the preamble to comprise …one processor, …one memory including program code,…the computer program code configured to, with the at least one processor, cause the apparatus to perform: the steps of the invention. therefore, the steps including instructions of a program to be performed by a processor of the apparatus. Therefore, what the propose prior arts provide are instructions that can be executed by a processor to implement a program which falls within the BRI scope of the claimed invention for the apparatus, and these different methods of Lee’s teachings concerning of different algorithms such as disclosed in Lee’s Abstract and section V.B, 1st par., hence are algorithms, instructions executed by a processor a computer system, section VI., 2nd Par., discloses “this algorithm…recent development of computer speed…”. Nevertheless, the invention, even when concerning different methods, all individually or individually together or together, can be understood to be a system or apparatus that performs analogous functions and steps of the recited limitations.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102 of this title, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claims 1-2, 4-6, 9-10, 12-13, 15-17, 20 and 22 are rejected under 35 U.S.C. 103 as being unpatentable over Jungjae Lee et. al. (“Measurement of 3D Spreader Position for Automatic Landing System, Nov. 2004, The 30th Annual Conference of the IEEE Industrial Electronics Society, Busan, Korea” hereinafter as “Lee”) in view of Heung-Yeung Shum et. al. (“Construction of Panoramic Image Mosaics with Global and Local Alignment, August 1999, 1999 Kluwer Academic Publisher, Printed in the Netherlands” hereinafter as “Shum”) and John T. Feddema et. al. (“US 5,785,191” hereinafter as “Feddema”).
Regarding claim 1, Lee explicitly teaches an apparatus comprising at least one processor; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the at least one processor, cause the apparatus to perform (Abstract discloses that the processing is done on images, which can be understood to have the use of a computer which includes a processor to execute program stored in a memory): receiving a first image of a first feature of a load (Section III, 1st Par., discloses having cameras to take pictures of two corners of a container [analogous to a load as claimed, by BRI/broadest reasonable interpretation], therefore, any of the pictures is analogous to the first image as claimed, and the any of the corners is analogous to the first feature as claimed); receiving a second image of a second feature of the load (Any of the other pictures is analogous to the second image as claimed, based on BRI, and any of the corners is analogous to the second feature as claimed); determining image plane coordinates of the features of the load based on the first image and the second image (Section III.A, 1st Par., discloses extracting plane coordinates of the corners based on the pictures taken from the two cameras, which is analogous to the claimed limitation); determining an operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature (Section V.B discloses determination of misalignment between the spreader and the container based on the coordinates found previously from the pictures to call misalignment error); determining one or more action candidates based on the operation (Section V.C discloses when the misalignment error is found, take an appropriate action such as moving the spreader to either right or left based on where the misalignment is found, which, by BRI, is analogous to the claimed limitation); evaluating the one or more action candidates using an intermediate medium embodying historical experience information within a finite time horizon to obtain cost(s) and/or reward(s) for the one or more action candidates (Section V.C shops different experiment results [also shown in tables I-III] to compensate for the misalignment errors found, the experience results are analogous to the historical experience information as claimed based on BRI, and the different methods here being analogous to the action candidates as claimed, by BRI; and the different method results here being applied according to cost such as if the system is too expensive, switch to using cheaper cameras, therefore, to obtain economical and essential and effectiveness of the system based on the experiment results [“and/or” indicates a selection, the examiner selects cost for mapping which is analogous to determining that when the system is too expensive, switch to using cheaper cameras for appropriate method to be used effectively]; since the movement and the camera taking picture continuously, therefore, the data in tables I-III can be understood to be in time series which is analogous to a finite time horizon as claimed, by BRI); and determining a control action based on the cost(s) and/or reward(s) of the action candidates, wherein the control action causes a spreader to move with respect to the load (Section V.C, when the method is appropriate to used based on cost effective and misalignment compensation effectiveness, the spreader is moved accordingly to compensate the misalignment error, hence, by BRI, covers the scope of the claim, the moving of the spreader here is analogous to the control action as claimed); and transmitting the control action to one or more actuators for moving the spreader with respect to the load (the control action is to move the spreader with respect to the load, hence, can be understood to transmit signal to carry out the action accordingly).
However, Lee does not explicitly teach the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
In the same field of misalignment determination (Page 27, last Par., Shum), Shum explicitly teaches the operation being a pairwise operation (Page 22, 1st Par., discloses using pairwise operation to detect misalignment between data from two images which is analogous to the determination of the misalignment between the spreader and the container from two images of Lee), wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour (“or” indicates a selection, the examiner selects “monotonic” for mapping which is disclosed in Shum’s Page 6, 3rd to the last Pars., wherein the operation being monotonic, which is analogous to the claimed limitation, by BRI).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee of having an apparatus to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining an operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the operation, with the teachings of Shum of having wherein the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
Wherein having Lee’s apparatus performs receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining a pairwise operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
The motivation behind the modification would have been to obtain an apparatus of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics. Since both Lee and Shum share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s system improve spreader navigation based on using 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, see Lee’s Abstract and Shum’s system performs image misalignment determination and compensation based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Shum’s Abstract.
However, Lee in view of Shum does not explicitly teach wherein the one or more action candidates are sample time independent; the control action is sample time independent.
In the same field of spreader container movement system (Title and Abstract, Feddema), Feddema explicitly teaches wherein the one or more action candidates are sample time independent (Col. 7, lines 37-67, discloses “the present invention represents the dynamics of the gantry-style crane control system…implemented as an input-shaping infinite impulse response filter…several embodiments of the input-shaping filter are presented herein. A simple input-shaping filter that modifies the reference input so that the residual vibrations of Linear Time Invariant systems are eliminated was” indicating different systems can be used for the spreader mechanism, which is analogous to the different experiments/methods of Lee which are analogous to the recited action candidates, wherein these systems are time-invariant which is analogous to time independent, hence, can be understood as sample time independent action candidates as claimed); the control action is sample time independent (Col. 8, lines 19-46, discloses “the various components or subassemblies of the control system…provides input signals to a conventional crane controller to control motors on crane to move a spreader” indicating the input shaping system is to control the crane to move a spreader, hence, the control is based on the linear time-invariant system hence, the control action is therefore including time-independent information/data, hence, is also sample time-independent).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee in view of Shum of having apparatus performs receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining a pairwise operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour, with the teachings of Feddema of having wherein the one or more action candidates are sample time independent; the control action is sample time independent.
Wherein having Lee’s apparatus to perform: receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; evaluating the one or more sample time independent action candidates using an intermediate medium embodying historical experience information within a finite time horizon to obtain cost(s) and/or reward(s) for the one or more sample time independent action candidates; and determining a sample time independent control action based on the cost(s) and/or reward(s) of the sample time independent action candidates, wherein the sample time independent control action causes a spreader to move with respect to the load; and transmitting the sample time independent control action to one or more actuators for moving the spreader with respect to the load.
The motivation behind the modification would have been to obtain an apparatus of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics and uses Linear Time Invariance system with IRR filter that can be updated in real-time using measurements from the spreader-container system so can compensate for modeling errors and external disturbance so that the system can perform efficiently. Since both Lee and Shum and Feddema share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s in view of Shum’s obtain an apparatus of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Lee’s Abstract and Shum’s Abstract, and uses Linear Time Invariance system with IRR filter that can be updated in real-time using measurements from the spreader-container system so can compensate for modeling errors and external disturbance so that the system can perform efficiently, see Feddema’s Abstract.
Regarding claim 2, Lee in view of Shum and Feddema explicitly teaches the apparatus of claim 1, wherein Lee explicitly teaches a self-exploring algorithm is used in evaluating the one or more action candidates (Lee, as discussed previously, Section V.C discloses using appropriate method being experimented on to use for moving the spreader to compensate for the misalignment error, the different methods being experimented on for testing of the model is here is analogous to a self-exploring algorithm as claimed, since any method requires a self-determining of the experiment to determine an appropriate result).
Regarding claim 4, Lee in view of Shum and Feddema explicitly teaches the apparatus of claim 1, wherein Lee explicitly teaches the one or more action candidates and control action are defined based on displacement to x-direction, displacement to y- direction and rotation (Lee, the misalignment as discussed above, to be moved according to the action candidates, moreover, the misalignment is defined based on x and y directions such as show in in tables I-III, and the angle [analogous to the rotation as claimed]).
Regarding claim 5, Lee in view of Shum and Feddema explicitly teaches the apparatus of claim 1, wherein Lee explicitly teaches the operator has a behaviour correlated with decreasing or increasing errors in alignment of the spreader and the load (The misalignment relates to the decreasing or increasing errors in alignment or the misalignment of the spreader and the container, such as shown in tables I-III); and wherein the operation is a pairwise symmetry operation; or the operator is a norm of dot or cross multiplication of error vectors in the first image and the second image; or the operator is a norm of dot or cross multiplication of feature position vectors in the first image and the second image (“or” indicates a selection, the examiner selects “pairwise symmetry operation” for mapping which is disclosed in Shum’s 10, 1st par., wherein the operation considered symmetry determined, which, by BRI covers the scope of the claim).
However, Lee does not explicitly teach the operator being a pairwise operator has monotonic behaviour, the operation being pairwise operation.
In the same field of misalignment determination (Page 27, last Par., Shum), Shum explicitly teaches the operation being a pairwise operation (Page 22, 1st Par., discloses using pairwise operation to detect misalignment between data from two images which is analogous to the determination of the misalignment between the spreader and the container from two images of Lee), the operator being a pairwise operator has monotonic behaviour (“or” indicates a selection, the examiner selects “monotonic” for mapping which is disclosed in Shum’s Page 6, 3rd to the last Pars., wherein the operation being monotonic, which is analogous to the claimed limitation, by BRI).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee of having an apparatus to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; an operator has a behaviour correlated with decreasing or increasing errors in alignment of the spreader and the load; and wherein the operation is a pairwise symmetry operation; or the operator is a norm of dot or cross multiplication of error vectors in the first image and the second image; or the operator is a norm of dot or cross multiplication of feature position vectors in the first image and the second image, with the teachings of Shum of having wherein the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
Wherein having Lee’s apparatus to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; an operator has a behaviour correlated with decreasing or increasing errors in alignment of the spreader and the load; and wherein the operation is a pairwise symmetry operation; or the operator is a norm of dot or cross multiplication of error vectors in the first image and the second image; or the operator is a norm of dot or cross multiplication of feature position vectors in the first image and the second image; wherein the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
The motivation behind the modification would have been to obtain an apparatus of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics. Since both Lee and Shum share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s system improve spreader navigation based on using 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, see Lee’s Abstract and Shum’s system performs image misalignment determination and compensation based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Shum’s Abstract.
Regarding claim 6, Lee in view of Shum and Feddema explicitly teaches the apparatus of claim 1, wherein Lee explicitly teaches at least one reward for the one or more action candidates is obtained, and wherein at least one of said reward(s) achieves its highest value when the spreader substantially aligns with the load or achieves substantial alignment in the finite time horizon in the future (Lee’s section V, the method that provides the highest economical and effectiveness is selected for aligning the spreader with the container [“or” indicates a selection, the examiner selects “substantially aligns with the load”]).
Regarding claim 9, Lee in view of Shum and Feddema explicitly teaches the apparatus of claim 1, wherein Lee explicitly teaches the first image is received from a first camera located on a first corner of a spreader and the second image is received from a second camera located on a second corner of the spreader (The cameras to capture images of the corners of the container located on the corners of the spreader hence, wherein any of the cameras can be understood to be the first camera located on a first corner of the spreader and any of the others can be understood to be the second camera located on the second corner), wherein the first corner and the second corner are different corners, and wherein the first feature of the load is a first corner of a container and the second feature of the load is a second corner of the container, wherein the first corner of the spreader and the first corner of the container are corresponding corners and the second corner of the spreader and the second corner of the container are corresponding corners (The images captured each corners of the container, hence is analogous to any of the corner can be understood to be the first corner of the container and any of the others to be the second corner of the container, and the first feature being the first corner and the second feature being the second corner, and by BRI, the first corner location of the camera correspond to the first corner of the container and same for the second corner location of the camera correspond to the second corner of the container).
Regarding claim 10, Lee in view of Shum and Feddema explicitly teaches the apparatus of claim 1, wherein Lee explicitly teaches the first image is received from a first camera located on a first corner of a spreader and the second image is received from a second camera located on a second corner of the spreader, wherein the first corner and the second corner are different corners (Lee, the cameras to capture images of the corners of the container located on the corners of the spreader hence, by BRI, covers the scope of the claim, wherein any of the cameras can be understood to be the first camera located on a first corner of the spreader and any of the others can be understood to be the second camera located on the second corner), and wherein the first feature of the load is a first corner of a container and the second feature of the load is a second corner of the container, wherein the first corner of the spreader and the first corner of the container are corresponding corners and the second corner of the spreader and the second corner of the container are corresponding corners (Lee, the images captured each corners of the container, hence, by BRI, any of the corner can be understood to be the first corner of the container and any of the others to be the second corner of the container, and the first feature being the first corner and the second feature being the second corner, and by BRI, the first corner location of the camera correspond to the first corner of the container and same for the second corner location of the camera correspond to the second corner of the container); wherein the apparatus is further caused to perform: receiving a third image of a third feature of the load, wherein the third image is received from a third camera located on a third corner of the spreader; receiving a fourth image of a fourth feature of the load, wherein the fourth image is received from a fourth camera located on the fourth corner of the spreader (Lee, as shown in FIG. 10 and Fig. 11, the spreader system can include more than one spreader, hence here, the whole system of the experimental crane and the program environment can be understood to be the spreader as claimed, therefore, each spreader can be understood to carry two cameras at their corners, therefore, another spreader’s any of the set of cameras can be understood to be the capturing of a third image of a third corner/feature of the corresponding container [the load], and the other camera to be the fourth camera capturing a fourth image comprises a fourth corner/feature of the container [the load], by BRI, covers the scope of the claimed limitation); wherein the third corner and the fourth corner are different corners than the first corner and the second corner (Lee, since they are of a different spreader in the spreader system, they are different corners); and wherein the third feature of the load is a third corner of the container and the fourth feature of the load is a fourth corner of the container, wherein the third corner of the spreader and the third corner of the container are corresponding corners and the fourth corner of the spreader and the fourth corner of the container are corresponding corners (Lee, the third feature being the third corner and the fourth feature being the fourth corner and that the third corner of the spreader correspond to the third corner of the container and the fourth corner of the spreader corresponds to the fourth corner of the container, by BRI); and the apparatus further comprises means for determining image plane coordinates of the third and fourth features of the load based on the third image and the fourth image (section III.A, 1st Par., discloses extracting plane coordinates of the corners based on the pictures taken from the two cameras, which is analogous to the claimed limitation); determining another operation between the image plane coordinates of the third feature and the image plane coordinates of the fourth feature,; and determining one or more action candidates based on the operations (Section V.B discloses determination of misalignment between the spreader and the container based on the coordinates found previously from the pictures to call misalignment error).
However, Lee does not explicitly teach the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
In the same field of misalignment determination (page 27, last par., Shum), Shum explicitly teaches the operation being a pairwise operation (page 22, 1st par., discloses using pairwise operation to detect misalignment between data from two images which is analogous to the determination of the misalignment between the spreader and the container from two images of Lee), wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour (“or” indicates a selection, the examiner selects “monotonic” for mapping which is disclosed in Shum’s page 6, 3rd to the last par., wherein the operation being monotonic, which is analogous to the claimed limitation, by BRI).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee of having an apparatus to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; the first image is received from a first camera located on a first corner of a spreader and the second image is received from a second camera located on a second corner of the spreader, wherein the first corner and the second corner are different corners, with the teachings of Shum of having wherein the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
Wherein having Lee’s apparatus to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; the first image is received from a first camera located on a first corner of a spreader and the second image is received from a second camera located on a second corner of the spreader, wherein the first corner and the second corner are different corners; wherein the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
The motivation behind the modification would have been to obtain an apparatus of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics. Since both Lee and Shum share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s system improve spreader navigation based on using 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, see Lee’s Abstract and Shum’s system performs image misalignment determination and compensation based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Shum’s Abstract.
Regarding claim 12, Lee discloses a method comprising: receiving a second image of a second feature of the load (Any of the other pictures is analogous to the second image as claimed, based on BRI, and any of the corners is analogous to the second feature as claimed); determining image plane coordinates of the features of the load based on the first image and the second image (Section III.A, 1st par., discloses extracting plane coordinates of the corners based on the pictures taken from the two cameras, which is analogous to the claimed limitation); determining an operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature (Section V.B discloses determination of misalignment between the spreader and the container based on the coordinates found previously from the pictures to call misalignment error); determining one or more action candidates based on the operation (Section V.C discloses when the misalignment error is found, take an appropriate action such as moving the spreader to either right or left based on where the misalignment is found, which, by BRI, is analogous to the claimed limitation); evaluating the one or more action candidates using an intermediate medium embodying historical experience information within a finite time horizon to obtain cost(s) and/or reward(s) for the one or more action candidates (Section V.C shops different experiment results [also shown in tables I-III] to compensate for the misalignment errors found, the experience results are analogous to the historical experience information as claimed based on BRI, and the different methods here being analogous to the action candidates as claimed, by BRI; and the different method results here being applied according to cost such as if the system is too expensive, switch to using cheaper cameras, therefore, to obtain economical and essential and effectiveness of the system based on the experiment results [“and/or” indicates a selection, the examiner selects cost for mapping which is analogous to determining that when the system is too expensive, switch to using cheaper cameras for appropriate method to be used effectively]; since the movement and the camera taking picture continuously, therefore, the data in tables I-III can be understood to be in time series which is analogous to a finite time horizon as claimed, by BRI); and determining a control action based on the cost(s) and/or reward(s) of the action candidates, wherein the control action causes a spreader to move with respect to the load (Section V.C, when the method is appropriate to used based on cost effective and misalignment compensation effectiveness, the spreader is moved accordingly to compensate the misalignment error, hence, by BRI, covers the scope of the claim, the moving of the spreader here is analogous to the control action as claimed) ; and transmitting the control action to one or more actuators for moving the spreader with respect to the load (The control action is to move the spreader with respect to the load, hence, can be understood to transmit signal to carry out the action accordingly).
However, Lee does not explicitly teach the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
In the same field of misalignment determination (Page 27, last Par., Shum), Shum explicitly teaches the operation being a pairwise operation (Page 22, 1st Par., discloses using pairwise operation to detect misalignment between data from two images which is analogous to the determination of the misalignment between the spreader and the container from two images of Lee), wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour (“or” indicates a selection, the examiner selects “monotonic” for mapping which is disclosed in Shum’s Page 6, 3rd to the last Pars., wherein the operation being monotonic, which is analogous to the claimed limitation, by BRI).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee of having method to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining an operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the operation, with the teachings of Shum of having wherein the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
Wherein having Lee’s method to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining a pairwise operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
The motivation behind the modification would have been to obtain an method of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics. Since both Lee and Shum share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s system improve spreader navigation based on using 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, see Lee’s Abstract and Shum’s system performs image misalignment determination and compensation based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Shum’s Abstract.
However, Lee in view of Shum does not explicitly teach wherein the one or more action candidates are sample time independent; the control action is sample time independent.
In the same field of spreader container movement system (Title and Abstract, Feddema), Feddema explicitly teaches wherein the one or more action candidates are sample time independent (Col. 7, lines 37-67, discloses “the present invention represents the dynamics of the gantry-style crane control system…implemented as an input-shaping infinite impulse response filter…several embodiments of the input-shaping filter are presented herein. A simple input-shaping filter that modifies the reference input so that the residual vibrations of Linear Time Invariant systems are eliminated was” indicating different systems can be used for the spreader mechanism, which is analogous to the different experiments/methods of Lee which are analogous to the recited action candidates, wherein these systems are time-invariant which is analogous to time independent, hence, can be understood as sample time independent action candidates as claimed); the control action is sample time independent (Col. 8, lines 19-46, discloses “the various components or subassemblies of the control system…provides input signals to a conventional crane controller to control motors on crane to move a spreader” indicating the input shaping system is to control the crane to move a spreader, hence, the control is based on the linear time-invariant system hence, the control action is therefore including time-independent information/data, hence, is also sample time-independent).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee in view of Shum of having a method to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining a pairwise operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour, with the teachings of Feddema of having wherein the one or more action candidates are sample time independent; the control action is sample time independent.
Wherein having Lee’s method to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; evaluating the one or more sample time independent action candidates using an intermediate medium embodying historical experience information within a finite time horizon to obtain cost(s) and/or reward(s) for the one or more sample time independent action candidates; and determining a sample time independent control action based on the cost(s) and/or reward(s) of the sample time independent action candidates, wherein the sample time independent control action causes a spreader to move with respect to the load; and transmitting the sample time independent control action to one or more actuators for moving the spreader with respect to the load.
The motivation behind the modification would have been to obtain a method of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics and uses Linear Time Invariance system with IRR filter that can be updated in real-time using measurements from the spreader-container system so can compensate for modeling errors and external disturbance so that the system can perform efficiently. Since both Lee and Shum and Feddema share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s in view of Shum’s obtain an apparatus of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Lee’s Abstract and Shum’s Abstract, and uses Linear Time Invariance system with IRR filter that can be updated in real-time using measurements from the spreader-container system so can compensate for modeling errors and external disturbance so that the system can perform efficiently, see Feddema’s Abstract.
Regarding claim 13, Lee in view of Shum and Feddema explicitly teaches the method of claim 12, wherein Lee explicitly teaches a self-exploring algorithm is used in evaluating the one or more action candidates (Lee, section V.C discloses using appropriate method being experimented on to use for moving the spreader to compensate for the misalignment error, the different methods being experimented on for testing of the model is here is analogous to a self-exploring algorithm as claimed, since any method requires a self-determining of the experiment to determine an appropriate result).
Regarding claim 15, Lee in view of Shum and Feddema explicitly teaches the method of claim 12, wherein Lee explicitly teaches the one or more action candidates and control action are defined based on displacement to x-direction, displacement to y- direction and rotation (Lee, the misalignment as discussed above, to be moved according to the action candidates, which are already mapped in claim 12 above, moreover, the misalignment is defined based on x and y directions such as show in in tables I-III, and the angle [analogous to the rotation as claimed]).
Regarding claim 16, Lee in view of Shum and Feddema explicitly teaches the method of claim 12, wherein Lee explicitly teaches the operator has a behaviour (As discussed above in claim 12, as being mapped to Shum) correlated with decreasing or increasing errors in alignment of the spreader and the load (As discussed above in claim 12, the misalignment relates to the decreasing or increasing errors in alignment or the misalignment of the spreader and the container, such as shown in tables I-III); and wherein the operation is a pairwise symmetry operation; or the operator is a norm of dot or cross multiplication of error vectors in the first image and the second image; or the operator is a norm of dot or cross multiplication of feature position vectors in the first image and the second image (“or” indicates a selection, the examiner selects “pairwise symmetry operation” for mapping which is disclosed in Shum’s 10, 1st par., wherein the operation considered symmetry determined, which, by BRI covers the scope of the claim).
However, Lee does not explicitly teach the operator being a pairwise operator has monotonic behaviour, the operation being pairwise operation.
In the same field of misalignment determination (Page 27, last Par., Shum), Shum explicitly teaches the operation being a pairwise operation (Page 22, 1st Par., discloses using pairwise operation to detect misalignment between data from two images which is analogous to the determination of the misalignment between the spreader and the container from two images of Lee), the operator being a pairwise operator has monotonic behaviour (“or” indicates a selection, the examiner selects “monotonic” for mapping which is disclosed in Shum’s Page 6, 3rd to the last Pars., wherein the operation being monotonic, which is analogous to the claimed limitation, by BRI).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee of having a method to perform: receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining an operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the operation, with the teachings of Shum of having wherein the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
Wherein having Lee’s method performs receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining a pairwise operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
The motivation behind the modification would have been to obtain a method of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics. Since both Lee and Shum share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s system improve spreader navigation based on using 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, see Lee’s Abstract and Shum’s system performs image misalignment determination and compensation based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Shum’s Abstract.
Regarding claim 17, Lee in view of Shum and Feddema explicitly teaches the method of claim 12, wherein Lee explicitly teaches at least one reward for the one or more action candidates is obtained, and wherein at least one said reward(s) achieves its highest value when the spreader substantially aligns with the load or achieves substantial alignment in the finite time horizon in the future (Lee’s section V, the method that provides the highest economical and effectiveness is selected for aligning the spreader with the container [“or” indicates a selection, the examiner selects “substantially aligns with the load”]).
Regarding claim 20, Lee in view of Shum and Feddema explicitly teaches the method of claim 12, wherein Lee explicitly teaches the first image is received from a first camera located on a first corner of a spreader and the second image is received from a second camera located on a second corner of the spreader (The cameras to capture images of the corners of the container located on the corners of the spreader hence, by BRI, covers the scope of the claim, wherein any of the cameras can be understood to be the first camera located on a first corner of the spreader and any of the others can be understood to be the second camera located on the second corner), wherein the first corner and the second corner are different corners, and wherein the first feature of the load is a first corner of a container and the second feature of the load is a second corner of the container, wherein the first corner of the spreader and the first corner of the container are corresponding corners and the second corner of the spreader and the second corner of the container are corresponding corners (The images captured each corners of the container, hence, by BRI, any of the corner can be understood to be the first corner of the container and any of the others to be the second corner of the container, and the first feature being the first corner and the second feature being the second corner, and by BRI, the first corner location of the camera correspond to the first corner of the container and same for the second corner location of the camera correspond to the second corner of the container).
Regarding claim 22, Lee explicitly teaches a non-transitory computer readable medium comprising program instructions that, when executed by at least one processor, cause an apparatus to perform at least (Abstract discloses that the processing is done on images, which can be understood to have the use of a computer which includes a processor to execute program stored in a memory or a non-transitory computer readable medium such as a ROM or RAM): receiving a first image of a first feature of a load (Section III, 1st par., discloses having cameras to take pictures of two corners of a container [analogous to a load as claimed, by BRI/broadest reasonable interpretation], therefore, any of the pictures is analogous to the first image as claimed, and the any of the corners is analogous to the first feature as claimed); receiving a second image of a second feature of the load (Any of the other pictures is analogous to the second image as claimed, based on BRI, and any of the corners is analogous to the second feature as claimed); determining image plane coordinates of the features of the load based on the first image and the second image (Section III.A, 1st par., discloses extracting plane coordinates of the corners based on the pictures taken from the two cameras, which is analogous to the claimed limitation); determining an operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature (Section V.B discloses determination of misalignment between the spreader and the container based on the coordinates found previously from the pictures to call misalignment error); determining one or more action candidates based on the operation (Section V.C discloses when the misalignment error is found, take an appropriate action such as moving the spreader to either right or left based on where the misalignment is found, which, by BRI, is analogous to the claimed limitation); evaluating the one or more action candidates using an intermediate medium embodying historical experience information within a finite time horizon to obtain cost(s) and/or reward(s) for the one or more action candidates (Section V.C shops different experiment results [also shown in tables I-III] to compensate for the misalignment errors found, the experience results are analogous to the historical experience information as claimed based on BRI, and the different methods here being analogous to the action candidates as claimed, by BRI; and the different method results here being applied according to cost such as if the system is too expensive, switch to using cheaper cameras, therefore, to obtain economical and essential and effectiveness of the system based on the experiment results [“and/or” indicates a selection, the examiner selects cost for mapping which is analogous to determining that when the system is too expensive, switch to using cheaper cameras for appropriate method to be used effectively]; since the movement and the camera taking picture continuously, therefore, the data in tables I-III can be understood to be in time series which is analogous to a finite time horizon as claimed, by BRI); and determining a control action based on the cost(s) and/or reward(s) of the action candidates, wherein the control action causes a spreader to move with respect to the load (Section V.C, when the method is appropriate to used based on cost effective and misalignment compensation effectiveness, the spreader is moved accordingly to compensate the misalignment error, hence, by BRI, covers the scope of the claim, the moving of the spreader here is analogous to the control action as claimed) ; and transmitting the control action to one or more actuators for moving the spreader with respect to the load (The control action is to move the spreader with respect to the load, hence, can be understood to transmit signal to carry out the action accordingly).
However, Lee does not explicitly teach the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
In the same field of misalignment determination (Page 27, last Par., Shum), Shum explicitly teaches the operation being a pairwise operation (Page 22, 1st Par., discloses using pairwise operation to detect misalignment between data from two images which is analogous to the determination of the misalignment between the spreader and the container from two images of Lee), wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour (“or” indicates a selection, the examiner selects “monotonic” for mapping which is disclosed in Shum’s Page 6, 3rd to the last Pars., wherein the operation being monotonic, which is analogous to the claimed limitation, by BRI).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee of having a system to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining an operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the operation, with the teachings of Shum of having wherein the operation being a pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
Wherein having Lee’s system performs receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining a pairwise operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour.
The motivation behind the modification would have been to obtain a system of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics. Since both Lee and Shum share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s system improve spreader navigation based on using 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, see Lee’s Abstract and Shum’s system performs image misalignment determination and compensation based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Shum’s Abstract.
However, Lee in view of Shum, in combination, does not explicitly teach wherein the one or more action candidates are sample time independent; the control action is sample time independent.
In the same field of spreader container movement system (Title and Abstract, Feddema), Feddema explicitly teaches wherein the one or more action candidates are sample time independent (Col. 7, lines 37-67, discloses “the present invention represents the dynamics of the gantry-style crane control system…implemented as an input-shaping infinite impulse response filter…several embodiments of the input-shaping filter are presented herein. A simple input-shaping filter that modifies the reference input so that the residual vibrations of Linear Time Invariant systems are eliminated was” indicating different systems can be used for the spreader mechanism, which is analogous to the different experiments/methods of Lee which are analogous to the recited action candidates, wherein these systems are time-invariant which is analogous to time independent, hence, can be understood as sample time independent action candidates as claimed); the control action is sample time independent (Col. 8, lines 19-46, discloses “the various components or subassemblies of the control system…provides input signals to a conventional crane controller to control motors on crane to move a spreader” indicating the input shaping system is to control the crane to move a spreader, hence, the control is based on the linear time-invariant system hence, the control action is therefore including time-independent information/data, hence, is also sample time-independent).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee in view of Shum of having a system performs receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; determining a pairwise operation between the image plane coordinates of the first feature and the image plane coordinates of the second feature; determining one or more action candidates based on the pairwise operation, wherein a pairwise operator of the pairwise operation has a monotonic or piecewise monotonic behaviour, with the teachings of Feddema of having wherein the one or more action candidates are sample time independent; the control action is sample time independent.
Wherein having Lee’s system to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; evaluating the one or more sample time independent action candidates using an intermediate medium embodying historical experience information within a finite time horizon to obtain cost(s) and/or reward(s) for the one or more sample time independent action candidates; and determining a sample time independent control action based on the cost(s) and/or reward(s) of the sample time independent action candidates, wherein the sample time independent control action causes a spreader to move with respect to the load; and transmitting the sample time independent control action to one or more actuators for moving the spreader with respect to the load.
The motivation behind the modification would have been to obtain a system of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics and uses Linear Time Invariance system with IRR filter that can be updated in real-time using measurements from the spreader-container system so can compensate for modeling errors and external disturbance so that the system can perform efficiently. Since both Lee and Shum and Feddema share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee’s in view of Shum’s obtain an apparatus of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics, see Lee’s Abstract and Shum’s Abstract, and uses Linear Time Invariance system with IRR filter that can be updated in real-time using measurements from the spreader-container system so can compensate for modeling errors and external disturbance so that the system can perform efficiently, see Feddema’s Abstract.
Claims 7 and 18 are rejected under 35 U.S.C. 103 as being unpatentable over Jungjae Lee et. al. (“Measurement of 3D Spreader Position for Automatic Landing System, Nov. 2004, The 30th Annual Conference of the IEEE Industrial Electronics Society, Busan, Korea” hereinafter as “Lee”) in view of Heung-Yeung Shum et. al. (“Construction of Panoramic Image Mosaics with Global and Local Alignment, August 1999, 1999 Kluwer Academic Publisher, Printed in the Netherlands” hereinafter as “Shum”) further in view of John T. Feddema et. al. (“US 5,785,191” hereinafter as “Feddema”) and Yunfeng Diao et. al. (“Vision-based Detection of Container Lock Holes Using a Modified Local Sliding Window Method, June 2019, EURASIP Journal on Image and Video Processing, Vol. 2019, Article Number 69” hereinafter as “Diao”).
Regarding claim 7, Lee in view of Shum and Feddema explicitly teaches the apparatus of claim 1.
However, Lee in view of Shum and Feddema does not explicitly teach wherein at least one cost for the one or more action candidates is obtained, and wherein at least one of said cost(s) is proportional to force or energy or pressure or voltage or current or placement or placement consumption of the action candidates and their effect in the spreader motion at the current moment or in the finite time horizon in the future; and/or reflects risk of losing features in a camera's field of view at the current moment or in the finite time horizon in the future.
In the same field of taking photo of container from cameras on a spreader (Title and Abstract, Diao), Diao explicitly teaches wherein at least one cost for the one or more action candidates is obtained, and wherein at least one of said cost(s) is proportional to force or energy or pressure or voltage or current or placement or placement consumption of the action candidates and their effect in the spreader motion at the current moment or in the finite time horizon in the future; and/or reflects risk of losing features in a camera's field of view at the current moment or in the finite time horizon in the future (“and/or” indicates a selection, the examiner selects “the cost reflects risk of losing features in a camera’s field of view at the current moment” which is disclosed in section 2.1.2, the shooting situation of the camera is being considered to capture the container effectively, further discloses in section 2.1.3 wherein the location of the camera installation is considered to decrease the container errors, therefore, the correctness of angle detection position of the camera installation is the cost as claimed which reflect the risk of losing feature sin the image, by BRI, covers the scope of the claim).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee in view of Shum and Feddema of having an apparatus to perform: receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; evaluating the one or more sample time independent action candidates using an intermediate medium embodying historical experience information within a finite time horizon to obtain cost(s) and/or reward(s) for the one or more sample time independent action candidates; and determining a sample time independent control action based on the cost(s) and/or reward(s) of the sample time independent action candidates, wherein the sample time independent control action causes a spreader to move with respect to the load; and transmitting the sample time independent control action to one or more actuators for moving the spreader with respect to the load, with the teachings of Diao of having wherein at least one cost for the one or more action candidates is obtained, and wherein at least one of said cost(s) is proportional to force or energy or pressure or voltage or current or placement or placement consumption of the action candidates and their effect in the spreader motion at the current moment or in the finite time horizon in the future; and/or reflects risk of losing features in a camera's field of view at the current moment or in the finite time horizon in the future
Wherein having Lee’s an apparatus to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image, wherein at least one cost for the one or more action candidates is obtained, and wherein at least one of said cost(s) is proportional to force or energy or pressure or voltage or current or placement or placement consumption of the action candidates and their effect in the spreader motion at the current moment or in the finite time horizon in the future; and/or reflects risk of losing features in a camera's field of view at the current moment or in the finite time horizon in the future.
The motivation behind the modification would have been to obtain an apparatus of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics and uses Linear Time Invariance system with IRR filter that can be updated in real-time using measurements from the spreader-container system so can compensate for modeling errors and external disturbance so that the system can perform efficiently and to improve efficiency of container handing and automatic stations by using automatic recognizing and locating container information to align with desired region. Since both Lee and Shum and Feddema and Diao share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee in view of Shum and Feddema’s system improves spreader navigation based on using 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, see Lee’s Abstract, and to improve efficiency of container handing and automatic stations by using automatic recognizing and locating container information to align with desired region, see Diao’s Abstract.
Regarding claim 18, Lee in view of Shum discloses the method of claim 12.
However, Lee in view of Shum and Feddema does not explicitly teach wherein at least one cost for the one or more action candidates is obtained, and wherein at least one of said cost(s) is proportional to force or energy or pressure or voltage or current or placement or placement consumption of the action candidates and their effect in the spreader motion at the current moment or in the finite time horizon in the future; and/or reflects risk of losing features in a camera's field of view at the current moment or in the finite time horizon in the future.
In the same field of taking photo of container from cameras on a spreader (Title and Abstract, Diao), Diao explicitly teaches wherein at least one cost for the one or more action candidates is obtained, and wherein at least one of said cost(s) is proportional to force or energy or pressure or voltage or current or placement or placement consumption of the action candidates and their effect in the spreader motion at the current moment or in the finite time horizon in the future; and/or reflects risk of losing features in a camera's field of view at the current moment or in the finite time horizon in the future (“and/or” indicates a selection, the examiner selects “the cost reflects risk of losing features in a camera’s field of view at the current moment” which is disclosed in section 2.1.2, the shooting situation of the camera is being considered to capture the container effectively, further discloses in section 2.1.3 wherein the location of the camera installation is considered to decrease the container errors, therefore, the correctness of angle detection position of the camera installation is the cost as claimed which reflect the risk of losing feature sin the image, by BRI, covers the scope of the claim).
Therefore, it would have been obvious to one or ordinary skill in the art before the effective filing date of the claimed invention was made to combine the teaches of Lee in view of Shum and Feddema of having a method to perform: receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image; evaluating the one or more sample time independent action candidates using an intermediate medium embodying historical experience information within a finite time horizon to obtain cost(s) and/or reward(s) for the one or more sample time independent action candidates; and determining a sample time independent control action based on the cost(s) and/or reward(s) of the sample time independent action candidates, wherein the sample time independent control action causes a spreader to move with respect to the load; and transmitting the sample time independent control action to one or more actuators for moving the spreader with respect to the load, with the teachings of Diao of having wherein at least one cost for the one or more action candidates is obtained, and wherein at least one of said cost(s) is proportional to force or energy or pressure or voltage or current or placement or placement consumption of the action candidates and their effect in the spreader motion at the current moment or in the finite time horizon in the future; and/or reflects risk of losing features in a camera's field of view at the current moment or in the finite time horizon in the future
Wherein having Lee’s method to perform receiving a first image of a first feature of a load; receiving a second image of a second feature of the load; determining image plane coordinates of the features of the load based on the first image and the second image, wherein at least one cost for the one or more action candidates is obtained, and wherein at least one of said cost(s) is proportional to force or energy or pressure or voltage or current or placement or placement consumption of the action candidates and their effect in the spreader motion at the current moment or in the finite time horizon in the future; and/or reflects risk of losing features in a camera's field of view at the current moment or in the finite time horizon in the future.
The motivation behind the modification would have been to obtain a method of spreader movement navigation that provides coordinate information that uses 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, which is based on the use of pairwise operation that can register images accurately to improve quality of image mosaics and uses Linear Time Invariance system with IRR filter that can be updated in real-time using measurements from the spreader-container system so can compensate for modeling errors and external disturbance so that the system can perform efficiently and to improve efficiency of container handing and automatic stations by using automatic recognizing and locating container information to align with desired region. Since both Lee and Shum and Feddema and Diao share the same endeavor of systems that perform image registration and alignment performance. Wherein Lee in view of Shum and Feddema’s system improves spreader navigation based on using 3D information to determine and compensate for skew and sway angles to reduce cost and enhance performance of such system, see Lee’s Abstract, and to improve efficiency of container handing and automatic stations by using automatic recognizing and locating container information to align with desired region, see Diao’s Abstract.
Pertinent Prior Art(s)
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure:
Edward J. Sommer et. al., “US 5,339,962”, Col. 4, lines 50-67, discloses “the polling includes sampling for a predetermined sample time with the detectors…penetration characteristics of the material portions” which is analogous to Lee’s methods or experiments to sample alignment between spreader and container based on certain detection method, wherein Sommer’s is to perform measurement of the container for injector, however, it can be understood to share the same endeavor of alignment a corresponding tool to match with the size of a container through some type of detection method, such as Lee’s method of matching the spreader to the container [See Lee’s section V.C, 1st Par.] and Sommer’s method of matching , of different time periods, being analogous to different action candidates; moreover, Sommer’s Col. 5, lines 16-45, discloses “an ignore time is counted from a time when a detection is made, so that although sampling takes place during this ignore time, the data is set aside for consideration only in special case” indicating that the sampling taking place includes a period of time that is sample time-independent, when time is ignored but sampling still takes place, hence is analogous to the recited “sample time-independent” for the action candidates); the control action is sample time independent (Col. 5, lines 16-45, discloses “it is also useful to ignore a portion of each item of material to be separated. Therefore, an ignore time is counted…which activates the individual air ejection mechanisms” indicating the control action being activating the ejection of material into the container, which is analogous to Lee’s spreader spreading into the container which is the control action to the mechanism based on the output of the corresponding previous processing of related information, therefore, the control action here is also sample time independent).
Jacobus, Charles J. et. al., “US 2018,0089616”, discloses an automated inventory management and material (or container) handling removes the requirement to operate fully automatically or all-manual using conventional task dedicated vertical storage and retrieval (S&R) machines. Inventory requests Automated vehicles plan their own movements to execute missions over a container yard, warehouse aisles or roadways, sharing this space with manually driven trucks. Automated units drive to planned speed limits, manage their loads (stability control), stop, go, and merge at intersections according human driving rules, use on-board sensors to identify static and dynamic obstacles, and human traffic, and either avoid them or stop until potential collision risk is removed. They identify, localize, and either pick-up loads (pallets, container, etc.) or drop them at the correctly demined locations. Systems without full automation can also implement partially automated operations (for instance load pick-up and drop), and can assure inherently safe manually operated vehicles (i.e., trucks that do not allow collisions).
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PHUONG HAU CAI whose telephone number is (571)272-9424. The examiner can normally be reached M-F 8:30 am - 5:00pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Chineyere Wills-Burns can be reached at (571) 272-9752. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PHUONG HAU CAI/Examiner, Art Unit 2673
/CHINEYERE WILLS-BURNS/Supervisory Patent Examiner, Art Unit 2673
Prosecution Insights