DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Remarks
This office action is responsive to the RCE filed on 07/01/2026.
Claim(s) 1-21 is/are pending in the application.
Independent claim(s) 1 was/were amended.
Dependent claim(s) 9 was/were amended.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 07/01/2026 has been entered.
Response to Arguments
Applicant's argument(s), regarding the amended portion(s) as recited in independent claim 1, filed 07/01/2026, have/has been fully considered and is/are persuasive. However, upon further consideration, a new ground(s) of rejection is made, adding/using Ramirez to be relied upon for the aforementioned amended portion(s). To note, applicant's amendment necessitated the new ground(s) of rejection presented in this office action.
Claim Rejections - 35 USC § 103
The text of those sections of Title 35, U.S. Code not included in this action can be found in a prior Office action.
Claim(s) 1, 7-10, 13-21 is/are rejected under 35 U.S.C. 103 as being unpatentable over Kroeze et al. (US 2021/0179172 A1) in view of Ramirez Llanos et al. (US 2022/0179429) (hereinafter Ramirez).
In regards to claim 1, Kroeze teaches a method for localizing a trailer in the surroundings of a towing vehicle, the method comprising:
reading in at least one single image in which the surroundings of the towing vehicle are imaged in two dimensions (e.g. [0057],Fig.4: once the towing operation has been determined, the method is then operative to receive 430 an image from a rear facing camera; the image may be a single image or a frame of a video stream; Examiner’s note: example shown in Fig.2a);
determining a feature representation from the at least one read-in single image using an image processing algorithm, wherein defined features of the trailer to be localized are reproduced in the feature representation (e.g. [0058],Fig.4: method is next operative to perform 435 image processing techniques on the received image to generate a processed image; in one exemplary embodiment, the image processing techniques may be edge detection techniques or other image processing techniques, similar to those used to generate the hitch assembly model; Examiner’s note: detected edges represent the feature representation; example shown in Fig.2d);
comparing the feature presentation with one or more of a plurality of trailer models (e.g. [0058],Fig.4: processed image is then compared 440 to the hitch assembly model to determine an angular displacement the hitch assembly elements detected in each image/model; the method is then operative to estimate an hitch articulation angle (HAA) in response to the angular displacement; see also [0047]: an initial learning routine may be started to learn an image featured model for each customer’s trailer regardless of additions to trailer; trailer templates of different hitch articulation angles may then be generated by applying view perspective transformation and image rotation; thus, a template matching step to determine a hitch articulation angle for may be performed for each rear view camera video frame; soft edge matching scores may then be calculated by comparing current image edge map with the learned templates; Examiner’s note: where the trailer templates represent the different hitch assembly models used to determine HAA), the comparison comprising:
modifying a model orientation and/or a model position and/or a model pose of a respective trailer model from the at least one read-in model dataset for by fitting the respective trailer model to the determined feature representation (e.g. [0046]: the HAA may be estimated by edge matching performed between images by comparison against rotated templates and wherein the HAA is determined by similarity match; as above, [0047]: a template matching step to determine a hitch articulation angle for may be performed for each rear view camera video frame; soft edge matching scores may then be calculated by comparing current image edge map with the learned templates; see also; Examiner’s note: this shows template models are rotated for comparison purposes in order to determine HAA); and
determining: the trailer orientation of the trailer to be localized is determined from the modified model orientation and/or the trailer position of the trailer to be localized from the modified model position and/or the trailer pose of the trailer to be localized from the modified model pose (e.g. as above, [0058],Fig.4: processed image is then compared 440 to the hitch assembly model to determine an angular displacement the hitch assembly elements detected in each image/model; the method is then operative to estimate an hitch articulation angle (HAA) in response to the angular displacement; Examiner’s note: this shows that orientation/position/pose would be determined),
but does not explicitly teach the method, comprising:
identifying a trailer model from a plurality of trailer models based on the determined feature representation by comparing the feature presentation with one or more of the plurality of trailer models, the comparison comprising:
reading in at least one model dataset from a trailer database that includes model datasets from the plurality of trailer models, wherein each model dataset a defined trailer model is simulated by a model; and
upon identifying the trailer model from the plurality of trailer models, determining: the trailer orientation of the trailer to be localized is determined from the modified model orientation and/or the trailer position of the trailer to be localized from the modified model position and/or the trailer pose of the trailer to be localized from the modified model pose.
However, Ramirez teaches a method, comprising:
identifying a trailer model from a plurality of trailer models based on the determined feature representation by comparing the feature presentation with one or more of the plurality of trailer models (e.g. [0037]: the controller 30 identifies the tow hitch 22 by comparing the two-dimensional image 40 of the tow hitch 22 taken by the camera 28 to the database 50 (Fig.5) having the predefined information about known tow hitches 22-D; [0038]: alternatively, the database 50 includes invariant features, such as histogram of oriented gradients (“HOG”), Color Names, RGB, and/or grayscale, to characterize the known tow hitches 22-D; the controller 30 utilizes the invariant features of the known tow hitches 22-D to compare to features of the tow hitch 22 from the image 40 to match with one of the known tow hitches 22-D from the database 50), the comparison comprising:
reading in at least one model dataset from a trailer database that includes model datasets from the plurality of trailer models, wherein each model dataset a defined trailer model is simulated by a model (e.g. as above, [0038]: the controller 30 utilizes the invariant features of the known tow hitches 22-D to compare to features of the tow hitch 22 from the image 40 to match with one of the known tow hitches 22-D from the database 50); and
upon identifying the trailer model from the plurality of trailer models, determining: the trailer orientation of the trailer to be localized is determined from the modified model orientation and/or the trailer position of the trailer to be localized from the modified model position and/or the trailer pose of the trailer to be localized from the modified model pose (e.g. [0040]-[0041]: when the model of the tow hitch 22 has been identified … the controller 30 produces a three-dimensional model tow hitch 22-3D; once the controller 30 has constructed the three-dimensional rendering 60 including the model tow hitch 22-3D, the controller 30 must position the model tow hitch 22-3D in the three-dimensional rendering 60 (step 140); with the model tow hitch 22-3D in the three-dimensional rendering 60, the model tow hitch 22-3D is orientated such that it corresponds to the orientation of the tow hitch 22 in the three-dimensional scene 44 when attached to the vehicle 20).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Kroeze to identify a specific trailer model, in the same conventional manner as taught by Ramirez as both deal with tow hitch positioning. The motivation to combine the two would be that it would allow the identification of the trailer model from a plurality of different trailer models.
In regards to processing unit claim 20, claim(s) 20 recite(s) limitations that is/are similar in scope to the limitations recited in claim 1. Therefore, claim(s) 20 is/are subject to rejections under the same rationale as applied hereinabove for claim 1. To note, paragraph [0048] discloses the use of a processor.
In regards to claim 7, the combination of Kroeze and Ramirez teaches a method, wherein an edge representation is determined as the feature representation, wherein at least trailer edges are reproduced in the edge representation as defined features of the trailer to be localized (e.g. Kroeze as above, [0058],Fig.4: perform 435 image processing techniques on the received image to generate a processed image; the image processing techniques may be edge detection techniques).
In regards to claim 8, the combination of Kroeze and Ramirez teaches a method, wherein the trailer edges of the edge representation are determined from at least one read-in single image using an edge algorithm (e.g. Kroeze as above, [0058],Fig.4: perform 435 image processing techniques on the received image to generate a processed image; the image processing techniques may be edge detection techniques).
In regards to claim 9, the combination of Kroeze and Ramirez teaches a method, wherein an edge-model dataset is read in as the model dataset, wherein the respective trailer model is described in the edge-model dataset by model edges which are characteristic for the simulated trailer model (e.g. Kroeze as above, [0047]: trailer templates of different hitch articulation angles may then be generated by applying view perspective transformation and image rotation; thus, a template matching step to determine a hitch articulation angle for may be performed for each rear view camera video frame; soft edge matching scores may then be calculated by comparing current image edge map with the learned templates).
In regards to claim 10, the combination of Kroeze and Ramirez teaches a method, wherein the trailer edges in the determined edge representation and/or the model edges in the read-in edge-model dataset are described two-dimensionally or three-dimensionally (e.g. Kroeze, [0055],Fig.4: generate 420 a model of the trailer hitch assembly; receive an image from a camera and to perform image recognition or image processing techniques to determine a physical model of the trailer hitch assembly; the image processing may include a perspective transformation or edge detection; Examiner’s note: example shown in Fig.2d, viewed as being in two-dimensions; model of trailer hitch assembly corresponding to the trailer templates generated).
In regards to claim 13, the combination of Kroeze and Ramirez teaches a method, wherein the simulated trailer model is represented in the respective at least one model dataset in scaled form (e.g. Kroeze as above, [0047]: trailer templates of different hitch articulation angles may then be generated by applying view perspective transformation and image rotation).
In regards to claim 14, the combination of Kroeze and Ramirez teaches a method, wherein the modification of a model orientation and/or a model position and/or a model pose of the respective trailer model is carried out by applying a geometric transformation to a particular model dataset of the at least one model datasets read in (e.g. Kroeze as above, [0046]: the HAA may be estimated by edge matching performed between images by comparison against rotated templates and wherein the HAA is determined by similarity match).
In regards to claim 15, the combination of Kroeze and Ramirez teaches a method, wherein the trailer orientation of the trailer to be localized and/or the trailer position of the trailer to be localized and/or the trailer pose of the trailer to be localized are determined from the geometric transformation, the application of which fits the respective trailer model to the determined feature representation (e.g. Kroeze as above, [0046]: the HAA may be estimated by edge matching performed between images by comparison against rotated templates and wherein the HAA is determined by similarity match).
In regards to claim 16, the combination of Kroeze and Ramirez teaches a method, wherein the fitting of the respective trailer model to the determined feature representation is carried out in a series of iteration steps, wherein the model orientation and/or the model position and/or the model pose of the respective trailer model is iteratively modified in the respective iteration steps (e.g. Kroeze as above, [0046]: the HAA may be estimated by edge matching performed between images by comparison against rotated templates and wherein the HAA is determined by similarity match; Examiner’s note: where rotation of different templates iteratively performed).
In regards to claim 17, the combination of Kroeze and Ramirez teaches a method, wherein the iterative fitting of the respective trailer model to the determined feature representation is terminated when an exit criterion is reached, wherein the exit criterion is satisfied when an iteration number of iteration steps has been reached and/or a mean distance between the trailer model of the respective model dataset and the respectively assigned features in the respective feature representation falls below a limit distance (e.g. Kroeze as above, [0046]: the HAA may be estimated by edge matching performed between images by comparison against rotated templates and wherein the HAA is determined by similarity match; Examiner’s note: similarity match shows limit distance determination).
In regards to claim 18, the combination of Kroeze and Ramirez teaches a method, wherein a chassis and/or a platform and/or a coupling are simulated as a model by means of the trailer model of the particular model dataset read in (e.g. Kroeze, [0055],Fig.4: generate 420 a model of the trailer hitch assembly; Examiner’s note: trailer hitch viewed as coupling).
In regards to claim 19, the combination of Kroeze and Ramirez teaches a method, wherein an articulation angle of the trailer relative to a towing vehicle and/or a trajectory for an approach of the towing vehicle to the trailer are determined from the determined trailer orientation and/or trailer position and/or trailer pose of the trailer (e.g. Kroeze as above, [0058],Fig.4: processed image is then compared 440 to the hitch assembly model to determine an angular displacement the hitch assembly elements detected in each image/model; the method is then operative to estimate an hitch articulation angle (HAA) in response to the angular displacement).
In regards to claim 21, Kroeze teaches a vehicle comprising the processing unit of claim 20 (e.g. [0048],Fig.3: system for determining a trailer hitch articulation angle in a motor vehicle 300; system 300 includes a processor 320, a vehicle controller 345, a trailer interface module 350, video controller 315, and a camera 310).
Claim(s) 2-3, 5-6 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kroeze and Ramirez as applied to claim 1 above, and further in view of Jin et al. (US 2021/0349217 A1).
In regards to claim 2, the combination of Kroeze and Ramirez teaches the method of claim 1 (Examiner’s note: Kroeze discloses in paragraph [0060] that the camera may be a LiDAR system), but does not explicitly teach the method, wherein a point cloud representation is determined as the feature representation, wherein the point cloud representation contains a point cloud of a plurality of object points in the surroundings, wherein at least some of the object points are assigned to the trailer in the surroundings.
However, Jin teaches a method, wherein a point cloud representation is determined as the feature representation, wherein the point cloud representation contains a point cloud of a plurality of object points in the surroundings, wherein at least some of the object points are assigned to the trailer in the surroundings (e.g. [0036]-[0038],Fig.2: at step 302, the multi-line LiDAR provided on each of the two sides of the tractor is controlled to emit laser light, such that a surface of the trailer reflects the laser light emitted by the multi-line LiDAR; at step 303, each of the multi-line LiDARs is controlled to receive a corresponding laser point cloud reflected by the surface of the trailer; at step 304, a trailer angle is calculated based on the corresponding laser point clouds received by the respective multi-line LiDARs and the initial point cloud data using a point cloud matching algorithm).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Kroeze and Ramirez to use point cloud as the feature representation, in the same conventional manner as taught by Jin as both deal with determining a trailer angle. The motivation to combine the two would be that it would allow the use of point cloud data for matching in order to determine trailer angle.
In regards to claim 3, the combination of Kroeze, Ramirez and Jin teaches a method, wherein in the point cloud representation at least the three-dimensional trailer shape of the trailer to be localized is reproduced as a defined feature (e.g. Jin as above, [0036]-[0038],Fig.2: at step 302, the multi-line LiDAR provided on each of the two sides of the tractor is controlled to emit laser light, such that a surface of the trailer reflects the laser light emitted by the multi-line LiDAR; at step 303, each of the multi-line LiDARs is controlled to receive a corresponding laser point cloud reflected by the surface of the trailer).
In addition, the same rationale/motivation of claim 2 is used for claim 3.
In regards to claim 5, the combination of Kroeze, Ramirez and Jin teaches a method, wherein a 3D model dataset is read in as the model dataset, wherein the respective trailer model is described in the 3D model dataset in three dimensions by means of model points (e.g. Jin as above, [0036]-[0038],Fig.2: at step 304, a trailer angle is calculated based on the corresponding laser point clouds received by the respective multi-line LiDARs and the initial point cloud data using a point cloud matching algorithm; Examiner’s note: shows matching with initial point cloud data (read-in model dataset)).
In addition, the same rationale/motivation of claim 2 is used for claim 5.
In regards to claim 6, the combination of Kroeze, Ramirez and Jin teaches a method, wherein the model points of the respective 3D model dataset are brought into overlap with the object points in the point cloud representation for fitting the respective trailer model to the determined point cloud representation (e.g. Jin as above, [0036]-[0038],Fig.2: at step 304, a trailer angle is calculated based on the corresponding laser point clouds received by the respective multi-line LiDARs and the initial point cloud data using a point cloud matching algorithm; Examiner’s note: point cloud matching shows determination of comparison/overlap).
In addition, the same rationale/motivation of claim 5 is used for claim 5.
Claim(s) 4 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kroeze, Ramirez and Jin as applied to claim 2 above, and further in view of Fan et al. (US 2015/0339541 A1).
In regards to claim 4, the combination of Kroeze, Ramirez and Jin teaches the method of claim 2, but does not explicitly teach the method, wherein the point cloud of the point cloud representation is determined using a Structure from Motion (SfM) algorithm, wherein depth information is determined by triangulation for a plurality of object points in the surroundings from at least two read-in single images using the SfM algorithm and the point cloud is generated from the plurality of object points as a function of the respectively determined depth information.
However, Fan teaches a method, wherein the point cloud of the point cloud representation is determined using a Structure from Motion (SfM) algorithm, wherein depth information is determined by triangulation for a plurality of object points in the surroundings from at least two read-in single images using the SfM algorithm and the point cloud is generated from the plurality of object points as a function of the respectively determined depth information (e.g. [0025]: in Structure-From-Motion (SFM), three-dimensional structures are estimated from two-dimensional image sequences, where the observer and/or the objects to be observed move in relation to each other; the obtained geometric models are stored as 3D point cloud; Examiner’s note: the three dimensions, including depth, would be estimated based on the different images).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Kroeze, Ramirez and Jin to determine point clouds, in the same conventional manner as taught by Fan as both deal with generating point cloud for point cloud matching. The motivation to combine the two would be that it would allow the generation of point clouds using multiple images.
Claim(s) 11-12 is/are rejected under 35 U.S.C. 103 as being unpatentable over the combination of Kroeze and Ramirez as applied to claim 1 above, and further in view of Ramamurthi et al. (US 2011/0231162 A1).
In regards to claim 11, the combination of Kroeze and Ramirez teaches the method of claim 1, but does not explicitly teach the method, wherein an intensity representation is determined as the feature representation by using an intensity algorithm, wherein in the intensity representation at least trailer intensity values are also reproduced in a spatially resolved manner as defined features of the trailer to be localized.
However, Ramamurthi teaches a method, wherein an intensity representation is determined as the feature representation by using an intensity algorithm, wherein in the intensity representation at least image intensity values are also reproduced in a spatially resolved manner as defined features of the image to be localized (e.g. [0023]: a pixel-by-pixel comparison of the images may be performed to identify differences in pixel intensity value, and a model may be determined to be a best fit match if the differences in pixel intensity value for the image(s) are below a pre-identified threshold).
Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to have modified the teachings/combination of Kroeze and Ramirez to model match, in the same conventional manner as taught by Ramamurthi as both deal with matching models based on feature representations. The motivation to combine the two would be that it would allow the matching of models based on corresponding spatially determined intensity values.
In regards to claim 12, the combination of Kroeze, Ramirez and Ramamurthi teaches a method, wherein an intensity-model dataset is read in as the model dataset, wherein the respective trailer model is described in the intensity-model dataset in a spatially resolved manner by model intensity values which are characteristic for the simulated trailer model (e.g. Ramamurthi as above, [0023]: a pixel-by-pixel comparison of the images may be performed to identify differences in pixel intensity value, and a model may be determined to be a best fit match if the differences in pixel intensity value for the image(s) are below a pre-identified threshold).
In addition, the same rationale/motivation of claim 11 is used for claim 12.
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JED-JUSTIN IMPERIAL whose telephone number is (571)270-5807. The examiner can normally be reached Monday to Friday, 9am - 6pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Daniel Hajnik can be reached at (571) 272-7642. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JED-JUSTIN IMPERIAL/Examiner, Art Unit 2616
/DANIEL F HAJNIK/Supervisory Patent Examiner, Art Unit 2616
Prosecution Insights