DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
This action is in response to the applicant’s communication filed on 9/30/2024
Claims 1-9 are pending.
Claim Interpretation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The claims in this application are given their broadest reasonable interpretation using the plain meaning of the claim language in light of the specification as it would be understood by one of ordinary skill in the art. The broadest reasonable interpretation of a claim element (also commonly referred to as a claim limitation) is limited by the description in the specification when 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is invoked.
As explained in MPEP § 2181, subsection I, claim limitations that meet the following three-prong test will be interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph:
(A) the claim limitation uses the term “means” or “step” or a term used as a substitute for “means” that is a generic placeholder (also called a nonce term or a non-structural term having no specific structural meaning) for performing the claimed function;
(B) the term “means” or “step” or the generic placeholder is modified by functional language, typically, but not always linked by the transition word “for” (e.g., “means for”) or another linking word or phrase, such as “configured to” or “so that”; and
(C) the term “means” or “step” or the generic placeholder is not modified by sufficient structure, material, or acts for performing the claimed function.
Use of the word “means” (or “step”) in a claim with functional language creates a rebuttable presumption that the claim limitation is to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites sufficient structure, material, or acts to entirely perform the recited function.
Absence of the word “means” (or “step”) in a claim creates a rebuttable presumption that the claim limitation is not to be treated in accordance with 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph. The presumption that the claim limitation is not interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, is rebutted when the claim limitation recites function without reciting sufficient structure, material or acts to entirely perform the recited function.
Claim limitations in this application that use the word “means” (or “step”) are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action. Conversely, claim limitations in this application that do not use the word “means” (or “step”) are not being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, except as otherwise indicated in an Office action.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are: “step information acquisition unit”, “comparison information acquisition unit”, “estimation unit”, “failure handling unit”, “failure detection unit”, and “appearance information acquisition unit” in claims 1-8.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of 35 U.S.C. 112(b):
(b) CONCLUSION.—The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the inventor or a joint inventor regards as the invention.
The following is a quotation of 35 U.S.C. 112 (pre-AIA ), second paragraph:
The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the subject matter which the applicant regards as his invention.
Claim 1 recites the limitation “the appearance of a moving object” in lines 2-3, and "the location and position" in lines 9-10. There is insufficient antecedent basis for this limitation in the claim.
Claim 2 recites the limitation “the step information acquired last time” in line 5, and "the step information acquired currently" in line 4. There is insufficient antecedent basis for this limitation in the claim.
Claim 7 recites the limitation "the sensors" in line 2. There is insufficient antecedent basis for this limitation in the claim.
Claim 8 recites the limitation "the appearance of a moving object" in line 4, and "the location and position" in lines 10-11. There is insufficient antecedent basis for this limitation in the claim.
Claim 9 recites the limitation "the appearance of a moving object" in line 2, and "the location and position" in lines 8. There is insufficient antecedent basis for this limitation in the claim.
The dependent claims are also rejected under 35 U.S.C. § 112 as they inherit all of the characteristics of the claim from which they depend and none of the dependent claims provide a cure for the indefiniteness of the parent claims.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
Claim(s) 1, 6, 8-9 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nordbruch USPGPUB 2017/0320529 A1 (hereinafter Nordbruch) in view of Haven et al. USPGPUB 2020/0134860 A1 (hereinafter Haven).
Regarding claim 1, Nordbruch teaches a system (Par. [0010], “a manufacturing system for manufacturing vehicles”) comprising:
a sensor configured to acquire an appearance information containing the appearance of a moving object movable by unmanned driving (Par. [0007], “the vehicle driving autonomously or remotely controlled within a manufacturing system for manufacturing vehicles.”; Par. [0027], “the driving operation of the vehicle is monitored and/or documented at least partially, in particular completely, with the aid of a vehicle-external monitoring system”; Par. [0034], “monitoring system includes one or multiple video camera(s) and/or one or multiple radar sensor(s) and/or one or multiple ultrasonic sensor(s) and/or one or multiple LIDAR sensor(s) and/or one or multiple laser sensor(s) and/or one or multiple photoelectric barrier(s)) and/or one or multiple door opening sensor(s)” – visual recording / camera or LIDAR monitoring of the vehicle during autonomous driving is interpreted as the appearance information containing the appearance of the moving object.);
a step information acquisition unit configured to acquire a step information about progress of a step of manufacturing the moving object (Par. [0021], “the assembly line includes a spatial sequence of assembly stations on which manufacturing steps are sequentially carried out on the vehicle to be manufactured”; Par. [0075], “the vehicle drives autonomously or remotely controlled from one production or manufacturing step A to another manufacturing step B”; Par. [0040], “target position data for one or multiple target(s) which the vehicle is supposed to drive to autonomously. A target of this type is, for example, a position or a location of an assembly station” – Nordbruch’s manufacturing system computer that acquires or provides target-position/assembly-station data corresponds to the step information acquisition unit, and received target position data corresponds to step information about progress of a step of manufacturing the moving object because the target assembly station or manufacturing step identifies where the vehicle is within the sequential manufacturing process).
Nordbruch does not explicitly teach a comparison information acquisition unit configured to acquire a comparison information representing the appearance of the moving object responsive to the step information; and
an estimation unit configured to estimate at least one of the location and position of the moving object by making a comparison between the comparison information and the appearance information.
However, Haven teaches a comparison information acquisition unit configured to acquire a comparison information representing the appearance of the moving object (Par. [0161], “The system may further comprise a CAD model of the part or subassembly wherein the control logic may convert the CAD model into the reference cloud.”; Par. [0101], “providing a reference cloud of 3D voxels which represent a reference surface of the reference part or sub assembly having the known reference pose.”; Par. [0120], “The step of providing may include the steps of providing a CAD model of the part or subassembly and converting the CAD model into the reference cloud.”; Par. [0123], “using at least one 2D/3D hybrid sensor to acquire a sample cloud of 3D voxels which represent a corresponding surface of a sample part or subassembly of the same type as the reference part or subassembly … the voxels of the sample and reference clouds are processed utilizing a matching algorithm to determine the pose of the sample part or subassembly”; Par. [0248], “CAD model of the part that completely represents the exterior surface of the part can be loaded into the memory of the solution and converted to a 3D cloud of voxels” – Nordbruch’s physical vehicle corresponds to the moving object. Control logic for converting the CAD model into the reference cloud corresponds to the comparison information acquisition unit, and the CAD-derived reference cloud corresponds to the comparison information representing the appearance of the moving object because the reference cloud represents the expected surface/appearance of the object used for comparison with sensor-acquired appearance information. In the proposed combination, a CAD model corresponding to Nordbruch’s vehicle is converted into a reference cloud for comparison with appearance information acquired from the physical vehicle.); and
an estimation unit configured to estimate at least one of the location and position of the moving object by making a comparison between the comparison information and the appearance information (Par. [0015], “The pose of an object is the position and orientation of the object in space relative to some reference position and orientation. The location of the object can be expressed in terms of X, Y, and Z.”; Par. [0123], “using at least one 2D/3D hybrid sensor to acquire a sample cloud of 3D voxels which represent a corresponding surface of a sample part or subassembly of the same type as the reference part or subassembly … the voxels of the sample and reference clouds are processed utilizing a matching algorithm to determine the pose of the sample part or subassembly.” – As applied to Nordbruch’s vehicle, Haven’s sample cloud represents the sensed surface/appearance of the vehicle, the reference cloud represents the reference surface/appearance of the vehicle, and the matching-based pose determination corresponds to estimating at least one of the location and position of the moving object by comparing the comparison information and the appearance information.).
It would have been obvious to acquire Haven’s CAD-derived reference cloud responsive to Nordbruch’s step information because Nordbruch teaches that the vehicle progresses through sequential assembly stations / manufacturing steps and receives target-position/assembly-station data for autonomous or remotely controlled driving. Since a vehicle’s expected appearance during manufacture predictably depends on the current manufacturing stage, a person of ordinary skill in the art would have selected or acquired the CAD-derived reference cloud corresponding to the vehicle’s current assembly station/manufacturing step so that the comparison information represents the expected appearance of the vehicle at that stage.
Nordbruch and Haven are analogous art because they are from the same field of endeavor and contain functional similarities. They both relate to vehicle manufacturing environments using sensor information to determine or use position information of an object.
Therefore, at the time of effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the above manufacturing system, as taught by Nordbruch, and incorporate a machine-vision technique for determining pose by comparing sensor-acquired sample-cloud information with reference-cloud/CAD-derived comparison information, as taught by Haven.
One of ordinary skill in the art would have been motivated to improve the accuracy and efficiency of determining the pose/location of Nordbruch’s vehicle during autonomous movement through the manufacturing system, as suggested by Haven (Par. [0099]).
Regarding claim 6, the combination of Nordbruch and Haven teaches all the limitations of the base claims as outlined above.
Nordbruch further teaches a step management device configured to manage manufacture of the moving object (Par. [0064], “manufacturing system 501 being configured to carry out a method for operating a manufacturing system for manufacturing vehicles, a vehicle carrying out the method for operating a vehicle with the aid of the manufacturing system within the scope of its manufacture.”; Par. [0020], “the manufacturing system includes an assembly line for vehicle manufacturing”; Par. [0021], “the assembly line includes a spatial sequence of assembly stations on which manufacturing steps are sequentially carried out on the vehicle to be manufactured”- Nordbruch’s manufacturing system 501 corresponds to the claimed step management device because it manages the vehicle-manufacturing environment, including the assembly line, assembly stations, and sequential manufacturing steps through which the vehicle proceeds during manufacture.), wherein
the step information acquisition unit acquires the step information from the step management device (Par. [0040], “target position data for one or multiple target(s) which the vehicle is supposed to drive to autonomously. A target of this type is, for example, a position or a location of an assembly station, a test facility, or an end of the assembly line”; Par. [0041], “The data which are relevant for the autonomous driving operation are preferably received by the vehicle or sent to the vehicle, e.g., with the aid of the manufacturing system, via the communication network … the manufacturing system sends these data to the vehicle”; Par. [0075], “the vehicle drives autonomously or remotely controlled from one production or manufacturing step A to another manufacturing step B” - Nordbruch’s target-position/assembly-station data corresponds to the step information because it identifies the assembly station or manufacturing step to which the vehicle is being directed during manufacture, and the vehicle-side component that receives the target-position/assembly-station data from the manufacturing system corresponds to the step information acquisition unit.
Regarding claim 8, Nordbruch teaches a device (Par. [0011], “a computer program is provided which includes program codes for carrying out the method for operating a vehicle and/or for operating a manufacturing system for manufacturing vehicles, when the computer program is executed on a computer”) comprising:
the appearance information being acquired by the sensor and containing the appearance of a moving object movable by unmanned driving (Par. [0007], “the vehicle driving autonomously or remotely controlled within a manufacturing system for manufacturing vehicles”; Par. [0027], “the driving operation of the vehicle is monitored and/or documented at least partially... with the aid of a vehicle-external monitoring system. … Documenting the driving operation includes in particular a visual recording”; Par. [0034], “monitoring system includes one or multiple video camera(s) and/or one or multiple radar sensor(s) and/or one or multiple ultrasonic sensor(s) and/or one or multiple LID AR sensor(s) and/or one or multiple laser sensor(s) and/or one or multiple photoelectric barrier(s)) and/or one or multiple door opening sensor(s)” – visual recording / camera or LIDAR monitoring of the vehicle during autonomous driving is interpreted as the appearance information containing the appearance of the moving object, and the vehicle monitoring system that receives the visual recording/monitoring data corresponds to the appearance information acquisition unit.); and
a step information acquisition unit configured to acquire a step information about progress of a step of manufacturing the moving object (Par. [0021], “the assembly line includes a spatial sequence of assembly stations on which manufacturing steps are sequentially carried out on the vehicle to be manufactured”; Par. [0075], “the vehicle drives autonomously or remotely controlled from one production or manufacturing step A to another manufacturing step B”; Par. [0040], “target position data for one or multiple target(s) which the vehicle is supposed to drive to autonomously. A target of this type is, for example, a position or a location of an assembly station” – Nordbruch’s manufacturing system computer that acquires or provides target-position/assembly-station data corresponds to the step information acquisition unit, and received target position data corresponds to step information about progress of a step of manufacturing the moving object because the target assembly station or manufacturing step identifies where the vehicle is within the sequential manufacturing process).
Nordbruch does not explicitly teach an appearance information acquisition unit configured to acquire an appearance information from a sensor;
a comparison information acquisition unit configured to acquire a comparison information representing the appearance of the moving object responsive to the step information; and
an estimation unit configured to estimate at least one of the location and position of the moving object by making a comparison between the comparison information and the appearance information.
However, Haven teaches an appearance information acquisition unit configured to acquire an appearance information from a sensor (Par. [0210] – [0211], “When an image is captured from each sensor, the pixel information, along with the depth information, is converted by a computer 13 (i.e. FIG. 2) into a collection of points in space, called a "point cloud". When an image is captured from each sensor, the pixel information, along with the depth information, is converted by a computer 13 (i.e. FIG. 2) into a collection of points in space, called a "point cloud"”; Par. [0212], “The computer 13 of FIG. 2 controls a controller which, in turn, controls a processor, a temperature controller, the camera 30, the emitter 32 and the detector 34 of the sensor 10.”);
a comparison information acquisition unit configured to acquire a comparison information representing the appearance of the moving object responsive to the step information (Par. [0161], “The system may further comprise a CAD model of the part or subassembly wherein the control logic may convert the CAD model into the reference cloud”; Par. [0101], “providing a reference cloud of 3D voxels which represent a reference surface of the reference part or subassembly having the known reference pose”; Par. [0120], “The step of providing may include the steps of providing a CAD model of the part or subassembly and converting the CAD model into the reference cloud.”; Par. [0123], “using at least one 2D/3D hybrid sensor to acquire a sample cloud of 3D voxels which represent a corresponding surface of a sample part or subassembly of the same type as the reference part or subassembly … the voxels of the sample and reference clouds are processed utilizing a matching algorithm to determine the pose of the sample part or subassembly”; Par. [0248], “CAD model of the part that completely represents the exterior surface of the part can be loaded into the memory of the solution and converted to a 3D cloud of voxels” - Nordbruch’s physical vehicle corresponds to the moving object. Control logic for converting the CAD model into the reference cloud corresponds to the comparison information acquisition unit, and the CAD-derived reference cloud corresponds to the comparison information representing the appearance of the moving object because the reference cloud represents the expected surface/appearance of the object used for comparison with sensor-acquired appearance information. In the proposed combination, a CAD model corresponding to Nordbruch’s vehicle is converted into a reference cloud for comparison with appearance information acquired from the physical vehicle.); and
an estimation unit configured to estimate at least one of the location and position of the moving object by making a comparison between the comparison information and the appearance information (Par. [0015], “The pose of an object is the position and orientation of the object in space relative to some reference position and orientation. The location of the object can be expressed in terms of X, Y, and Z.”; Par. [0123], “the voxels of the sample and reference clouds are processed utilizing a matching algorithm to determine the pose of the sample part or subassembly … the voxels of the sample and reference clouds are processed utilizing a matching algorithm to determine the pose of the sample part or subassembly” - As applied to Nordbruch’s vehicle, Haven’s sample cloud represents the sensed surface/appearance of the vehicle, the reference cloud represents the reference surface/appearance of the vehicle, and the matching-based pose determination corresponds to estimating at least one of the location and position of the moving object by comparing the comparison information and the appearance information).
It would have been obvious to acquire Haven’s CAD-derived reference cloud responsive to Nordbruch’s step information because Nordbruch teaches that the vehicle progresses through sequential assembly stations / manufacturing steps and receives target-position/assembly-station data for autonomous or remotely controlled driving. Since a vehicle’s expected appearance during manufacture predictably depends on the current manufacturing stage, a person of ordinary skill in the art would have selected or acquired the CAD-derived reference cloud corresponding to the vehicle’s current assembly station/manufacturing step so that the comparison information represents the expected appearance of the vehicle at that stage.
Nordbruch and Haven are analogous art because they are from the same field of endeavor and contain functional similarities. They both relate to vehicle manufacturing environments using sensor information to determine or use position information of an object.
Therefore, at the time of effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the above manufacturing system, as taught by Nordbruch, and incorporate a machine-vision technique for determining pose by comparing sensor-acquired sample-cloud information with reference-cloud/CAD-derived comparison information, as taught by Haven.
One of ordinary skill in the art would have been motivated to improve the accuracy and efficiency of determining the pose/location of Nordbruch’s vehicle during autonomous movement through the manufacturing system, as suggested by Haven (Par. [0099]).
Regarding claim 9, Nordbruch teaches a method (Par. [0005], “method for operating a manufacturing system as well as a computer program.”) comprising:
acquiring an appearance information containing the appearance of a moving object movable by unmanned driving (Par. [0007], “the vehicle driving autonomously or remotely controlled within a manufacturing system for manufacturing vehicles.”; Par. [0027], “the driving operation of the vehicle is monitored and/or documented at least partially, in particular completely, with the aid of a vehicle-external monitoring system”; Par. [0034], “monitoring system includes one or multiple video camera(s) and/or one or multiple radar sensor(s) and/or one or multiple ultrasonic sensor(s) and/or one or multiple LIDAR sensor(s) and/or one or multiple laser sensor(s) and/or one or multiple photoelectric barrier(s)) and/or one or multiple door opening sensor(s)” – visual recording / camera or LIDAR monitoring of the vehicle during autonomous driving is interpreted as the appearance information containing the appearance of the moving object.);
acquiring a step information about progress of a step of manufacturing the moving object (Par. [0021], “the assembly line includes a spatial sequence of assembly stations on which manufacturing steps are sequentially carried out on the vehicle to be manufactured”; Par. [0075], “the vehicle drives autonomously or remotely controlled from one production or manufacturing step A to another manufacturing step B.”; Par. [0040], “The data which are relevant for the autonomous driving operation are, for example, the following data, individually or in combination: … target position data for one or multiple target(s) which the vehicle is supposed to drive to autonomously. A target of this type is, for example, a position or a location of an assembly station, a test facility, or an end of the assembly line or a parking facility or a parking position in a parking facility.”; Par. [0041], “The data which are relevant for the autonomous driving operation are preferably received by the vehicle or sent to the vehicle, e.g., with the aid of the manufacturing system, via the communication network” – received target position data corresponds to step information about progress of a step of manufacturing the moving object because the target assembly station or manufacturing step identifies where the vehicle is within the sequential manufacturing process.).
Nordbruch does not explicitly teach acquiring a comparison information representing the appearance of the moving object responsive to the step information; and
estimating at least one of the location and position of the moving object by making a comparison between the comparison information and the appearance information.
However, Haven teaches acquiring a comparison information representing the appearance of the moving object responsive to the step information (Par. [0161], “The system may further comprise a CAD model of the part or subassembly wherein the control logic may convert the CAD model into the reference cloud.”; Par. [0101], “providing a reference cloud of 3D voxels which represent a reference surface of the reference part or sub assembly having the known reference pose.”; Par. [0120], “The step of providing may include the steps of providing a CAD model of the part or subassembly and converting the CAD model into the reference cloud.”; Par. [0123], “using at least one 2D/3D hybrid sensor to acquire a sample cloud of 3D voxels which represent a corresponding surface of a sample part or subassembly of the same type as the reference part or subassembly … the voxels of the sample and reference clouds are processed utilizing a matching algorithm to determine the pose of the sample part or subassembly”; Par. [0248], “CAD model of the part that completely represents the exterior surface of the part can be loaded into the memory of the solution and converted to a 3D cloud of voxels” – Nordbruch’s physical vehicle corresponds to the moving object. The CAD-derived reference cloud corresponds to the comparison information representing the appearance of the moving object because the reference cloud represents the expected surface/appearance of the object used for comparison with sensor-acquired appearance information. In the proposed combination, a CAD model corresponding to Nordbruch’s vehicle is converted into a reference cloud for comparison with appearance information acquired from the physical vehicle.); and
estimating at least one of the location and position of the moving object by making a comparison between the comparison information and the appearance information (Par. [0015], “The pose of an object is the position and orientation of the object in space relative to some reference position and orientation. The location of the object can be expressed in terms of X, Y, and Z.”; Par. [0123], “using at least one 2D/3D hybrid sensor to acquire a sample cloud of 3D voxels which represent a corresponding surface of a sample part or subassembly of the same type as the reference part or subassembly … the voxels of the sample and reference clouds are processed utilizing a matching algorithm to determine the pose of the sample part or subassembly.” – As applied to Nordbruch’s vehicle, Haven’s sample cloud represents the sensed surface/appearance of the vehicle, the reference cloud represents the reference surface/appearance of the vehicle, and the matching-based pose determination corresponds to estimating at least one of the location and position of the moving object by comparing the comparison information and the appearance information.).
It would have been obvious to acquire Haven’s CAD-derived reference cloud responsive to Nordbruch’s step information because Nordbruch teaches that the vehicle progresses through sequential assembly stations / manufacturing steps and receives target-position/assembly-station data for autonomous or remotely controlled driving. Since a vehicle’s expected appearance during manufacture predictably depends on the current manufacturing stage, a person of ordinary skill in the art would have selected or acquired the CAD-derived reference cloud corresponding to the vehicle’s current assembly station/manufacturing step so that the comparison information represents the expected appearance of the vehicle at that stage.
Nordbruch and Haven are analogous art because they are from the same field of endeavor and contain functional similarities. They both relate to vehicle manufacturing environments using sensor information to determine or use position information of an object.
Therefore, at the time of effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the above manufacturing system, as taught by Nordbruch, and incorporate a machine-vision technique for determining pose by comparing sensor-acquired sample-cloud information with reference-cloud/CAD-derived comparison information, as taught by Haven.
One of ordinary skill in the art would have been motivated to improve the accuracy and efficiency of determining the pose/location of Nordbruch’s vehicle during autonomous movement through the manufacturing system, as suggested by Haven (Par. [0099]).
Claim(s) 2 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nordbruch USPGPUB 2017/0320529 A1 (hereinafter Nordbruch) in view of Haven et al. USPGPUB 2020/0134860 A1 (hereinafter Haven), and further in view of Gallagher USPGPUB 2011/0060881 A1 (hereinafter Gallagher).
Regarding claim 2, the combination of Nordbruch and Haven teaches all the limitations of the base claims as outlined above.
Haven further teaches the sensor acquires the appearance information repeatedly (Par. [0088], “A hybrid 2D/3D sensor or camera acquires complete clouds of hundreds of thousands of 3D voxels thirty times per second.”).
Nordbruch further teaches the step information acquisition unit acquires the step information repeatedly (Par. [0075], “the vehicle drives autonomously or remotely controlled from one production or manufacturing step A to another manufacturing step B.”; Par. [0040]-[0041], “The data which are relevant for the autonomous driving operation are, for example, the following data, individually or in combination: … target position data for one or multiple target(s) which the vehicle is supposed to drive to autonomously. A target of this type is, for example, a position or a location of an assembly station, a test facility, or an end of the assembly line or a parking facility or a parking position in a parking facility. The data which are relevant for the autonomous driving operation are preferably received by the vehicle or sent to the vehicle, e.g., with the aid of the manufacturing system, via the communication network” - As the vehicle moves through different manufacturing steps, the target-position/assembly-station data is repeatedly received/acquired).
Nordbruch and Haven do not explicitly teach if a content of the step information acquired currently by the step information acquisition unit is the same as a content of the step information acquired last time, the comparison information acquisition unit does not acquire the comparison information responsive to the step information acquired currently, and
the estimation unit makes a comparison between the comparison information same as that used in a comparison made last time and the appearance information.
However, Gallagher teaches if a content of the step information acquired currently by the step information acquisition unit is the same as a content of the step information acquired last time, the comparison information acquisition unit does not acquire the comparison information responsive to the step information acquired currently (Par. [0018], “The data manager 107 includes a cache logic unit 204 that determines whether a data entry is to be retrieved from the main storage 104 or from the cache 106”; Par. [0025], “In response to the request, the data manager 107 determines whether the data entry can be located in the cache 105 block 320). If the data entry can be located in the cache 105, the data manager 107 determines whether the data entry is expiring or expired (block 330). If the data entry is neither expiring nor expired, the data manager 107 retrieves the data entry found in the cache 105 and returns the data entry to the requesting application (block 340).”), and
the estimation unit makes a comparison between the comparison information same as that used in a comparison made last time and the appearance information (Par. [0032], “the user makes a second authentication request for the same resource 60 seconds later. The requested data entry is found in the cache and a response is immediately returned by the cache.” - Gallagher’s reuse of cached data for a repeated same request corresponds to not reacquiring the same comparison information and instead using the previously acquired comparison information).
Nordbruch, Haven, and Gallagher are analogous art because they all contain functional similarities. They all relate to computer-based acquisition, retrieval, and processing of data used by an automated system.
Therefore, at the time of effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the above system for estimating the location/position of a moving vehicle in a manufacturing system, as taught by Nordbruch and Haven, and incorporate cache-based reuse of previously acquired comparison information when the current step information is the same as the previously acquired step information, as taught by Gallagher.
One of ordinary skill in the art would have been motivated improve data access time and improve the overall efficiency of the system, as suggested by Gallagher (Par. [0002] – [0003]).
Claim(s) 3-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nordbruch USPGPUB 2017/0320529 A1 (hereinafter Nordbruch) in view of Haven et al. USPGPUB 2020/0134860 A1 (hereinafter Haven), and further in view of Link et al. USPGPUB 2019/0258225 A1 (hereinafter Link).
Regarding claim 3, the combination of Nordbruch and Haven teaches all the limitations of the base claims as outlined above.
Haven further teaches mounting failure of a part at the moving object may be detected using a degree of match between the appearance of the moving object represented by the comparison information and the appearance of the moving object contained in the appearance information (Par. [0236], “The solution presents reference and sample clouds in a 3D display, allowing a human operator to review the degree of interpenetration of clouds from any viewpoint in 3D space … the aligned cloud represents the best fit of the sample cloud to the reference cloud”; Par. [0262], “The solution includes a 3D display that can be used to identify whether a single subassembly is shifted out of place. If the aligned cloud closely matches the reference cloud in all locations except in the location of one subassembly, then that subassembly can be quickly identified as being out of place with respect to the other subassemblies in the vehicle body.” — Haven’s 3D display for comparing the aligned sample cloud with the reference cloud corresponds to the claimed failure detection unit, Haven’s out of place subassembly in the vehicle body corresponds to mounting failure of a part at the moving object, and Haven’s determination that the aligned cloud closely matches the reference cloud except at one subassembly corresponds to detecting the mounting failure using a degree of match between the comparison information and the appearance information).
Nordbruch and Haven do not explicitly teach a failure detection unit configured to detect a defect using a degree of match between sensed information representing an inspected object and stored information representing the expected object.
However, Link teaches a failure detection unit configured to detect a defect using a degree of match between sensed information representing an inspected object and stored information representing the expected object (Par. [0163], “The defect detector 460 is configured to determine, based on comparisons between the laser module scans of an object being inspected and stored data files regarding expected dimensions of the object, whether the product is within certain tolerance levels. If the product is not within tolerance levels, the defect detector may issue a defect warning.”).
Nordbruch, Haven, and Link are analogous art because they are from the same field of endeavor and contain functional similarities. They all relate to computer-based systems using sensor information in a manufacturing or inspection environment to determine information about an object and response to that determined information.
Therefore, at the time of effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the above system for estimating the location/position of a moving vehicle in a manufacturing system, as taught by Nordbruch and Haven, and incorporate a failure handling unit configured to notify the occurrence of a mounting failure when the mounting failure is detected, as taught by Link.
Regarding claim 4, the combination of Nordbruch, Haven, and Link teaches all the limitations of the base claims as outlined above.
Link further teaches a failure handling unit configured to perform at least one of a process of stopping moving of the moving object and a process of notifying the occurrence of the mounting failure if the mounting failure is detected by the failure detection unit (Par. [0163], “The defect detector 460 is configured to determine, based on comparisons between the laser module scans of an object being inspected and stored data files regarding expected dimensions of the object, whether the product is within certain tolerance levels. If the product is not within tolerance levels, the defect detector may issue a defect warning.”; Par. [0164], “The part rejection unit 470 may receive information from the defect detector 460, and based on a determination of whether the defects meet predetermined quality threshold requirements, issue a command to stop the inspection process, flag the inspected object as defected … alert the operator of a defect condition through, for example, the alert notification 130, or the network interface 445.”; Par. [0165], “The transport motor control may also stop the transport 150 when a defective object is detected, allowing an operator time to remove the object from the inspection line or take other appropriate action” – The part rejection unit, transport motor control, and alert notification system corresponds to the failure handling unit.).
It would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention to further modify the Nordbruch-Haven system to include Link’s failure-handling process. Haven teaches identifying a shifted or out-of-place subassembly in a vehicle body by comparing the aligned sample cloud with the reference cloud, while Link teaches stopping transport/inspection or alerting an operator when a defective object is detected. A person of ordinary skill in the art would have been motivated to use Link’s stop/alert handling in the Nordbruch-Haven system so that, when Haven detects a mounting failure of a part on Nordbruch’s vehicle, the vehicle can be stopped and/or an operator can be notified for corrective action before the defective vehicle continues through the manufacturing process.
Claim(s) 5 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nordbruch USPGPUB 2017/0320529 A1 (hereinafter Nordbruch) in view of Haven et al. USPGPUB 2020/0134860 A1 (hereinafter Haven), and further in view of Cardenas Bernal USPGPUB 2018/0350055 A1 (hereinafter Bernal).
Regarding claim 5, the combination of Nordbruch and Haven teaches all the limitations of the base claims as outlined above.
Nordbruch and Haven do not explicitly teach a database containing association between the step information and the comparison information, wherein
the comparison information acquisition unit acquires the comparison information that is associated in the database with the step information acquired by the step information acquisition unit.
However, Bernal teaches a database containing association between the step information and the comparison information (Par. [0030], “at a particular assembly station, the AR device associated with the station is programmed for the part dedicated at that station … the object type is associated with a reference model and reference data prepared at 201”; Par. [0050], “the reference models for manufactured items are stored in a data store and each have an associated identifier, such as the part name or number” – As applied to Nordbruch and Haven’s system, Bernal’s assembly station/dedicated-part information corresponds to step information because it identifies the manufacturing stage and the object expected at that stage, and Berna’s reference model corresponds to comparison information because it provides the expected model data used for comparison with sensed appearance information.), wherein
the comparison information acquisition unit acquires the comparison information that is associated in the database with the step information acquired by the step information acquisition unit (Par. [0035], “At 211, a reference model of the object type is retrieved. For example, based on the object type identified at 203, a reference model corresponding to the object type is retrieved … the reference model is stored in a data store such as a database” – In the modified Nordbruch-Haven-Bernal system, retrieving Bernal’s stored reference model corresponding to the acquired station/part/object-type information corresponds to the comparison information acquisition unit acquiring the comparison information associated with the acquired step information.).
Nordbruch, Haven, and Bernal are analogous art because they are from the same field of endeavor and contain functional similarities. They all relate to computer-based systems using sensor information and reference information in a vehicle or manufacturing environment to determine information about a manufactured object.
Therefore, at the time of effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the above system for estimating the location/position of a moving vehicle in a manufacturing system, as taught by Nordbruch and Haven, and incorporate database-backed storage and retrieval of CAD-derived reference model information and corresponding assembly-sequence data, as taught by Bernal.
One of ordinary skill in the art would have been motivated to “increase the speed and efficiency related to manufacturing and in particular to the manufacturing of automobile parts and vehicles”, as suggested by Bernal (Par. [0002]).
Claim(s) 7 is/are rejected under 35 U.S.C. 103 as being unpatentable over Nordbruch USPGPUB 2017/0320529 A1 (hereinafter Nordbruch) in view of Haven et al. USPGPUB 2020/0134860 A1 (hereinafter Haven), and further in view of Overton et al. US 7,230,653 B1 (hereinafter Overton).
Regarding claim 7, the combination of Nordbruch and Haven teaches all the limitations of the base claims as outlined above.
Nordbruch further teaches a plurality of the sensors (Par. [0034], “the monitoring system includes one or multiple video camera(s) and/or one or multiple radar sensor(s) and/or one or multiple ultrasonic sensor(s) and/or one or multiple LIDAR sensor(s) and/or one or multiple laser sensor(s)”).
Nordbruch and Haven do not explicitly teach a database containing association between identification information about each of the sensors and the comparison information, wherein
the step information acquisition unit acquires the identification information about the sensor as the step information by which the appearance information has been acquired, and
the comparison information acquisition unit acquires the comparison information that is associated in the database with the identification information about the sensor by which the appearance information has been acquired.
However, Overton teaches a database containing association between identification information about each of the sensors and the comparison information (Col. 4, lines 44-50, “the controller 120 accesses at step 206 predefined image insertion rules in database 122 to determine, based at least in part on a camera identifier embedded in the telemetry data, what image or images-referred to herein as target images-are to be inserted into a particular video image in the frame of a video signal”; Col. 5, lines 8-9, “databases storing CAD models for the reference images and the target images” – camera identifier is interpreted as the identification information about each of the sensors, and CAD-based reference images is interpreted as the comparison information), wherein
the step information acquisition unit acquires the identification information about the sensor as the step information by which the appearance information has been acquired (Col. 4, lines 22-25, “Once video insertion system 100 receives an encoded video signal, a video/telemetry separator 118 extracts, as indicated by step 204, the telemetry data for a particular image within the video signal”; Col. 5, lines 66-67, “telemetry data indicates the identification, angle, focal distance and aperture setting of the camera taking the video image” – Overton’s telemetry data identifying the camera taking the video image corresponds to acquiring identification information about the sensor by which the appearance information has been acquired.), and
the comparison information acquisition unit acquires the comparison information that is associated in the database with the identification information about the sensor by which the appearance information has been acquired (Col. 4, lines 44-50, “the controller 120 accesses at step 206 predefined image insertion rules in database 122 to determine, based at least in part on a camera identifier embedded in the telemetry data, what image or images-referred to herein as target images-are to be inserted into a particular video image in the frame of a video signal” – Overton’s controller acquiring image information based on the camera identifier teaches acquiring corresponding reference image information associated with the sensor identification information. In the combined system, this teaching is applied to acquire Haven’s CAD/reference-cloud comparison information corresponding to the Nordbruch sensor that acquired the vehicle appearance information.).
Nordbruch, Haven, and Overton are analogous art because they contain functional similarities. They all relate to computer-based systems using sensor/camera information and stored model or reference information to process visual information from a current sensor/camera view.
Therefore, at the time of effective filing date, it would have been obvious to a person of ordinary skill in the art to modify the above system for estimating the location/position of a moving vehicle in a manufacturing system, as taught by Nordbruch and Haven, and incorporate selection or acquisition of CAD/model-based reference information based on identification information of the sensor/camera that acquired the image information, as taught by Overton.
One of ordinary skill in the art would have been motivated to reduce computational intensity and reduce costs, as suggested by Overton (Col. 1, lines 31-33; Col. 2, lines 7-11).
Citation of Pertinent Prior Art
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
Chen et al. [US 11,288,789 B1] teaches an image processing system and method obtains one or more source images in which a damaged vehicle is represented, and performs one or more image processing techniques on the obtained images to determine, predict, estimate, and/or detect damage that has occurred at various locations on the vehicle.
Graham et al. [US 2023/0018554 A1] teaches a method for inspecting an object includes determining a first inspection package that includes a first inspection image of the object and a first designation
Conclusion
Any inquiry concerning this communication or earlier communications from the examiner should be directed to PETER XU whose telephone number is (571)272-0792. The examiner can normally be reached Monday-Friday 9am-5pm.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Mohammad Ali can be reached at (571) 272-4105. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/PETER XU/ Examiner, Art Unit 2119
/MOHAMMAD ALI/Supervisory Patent Examiner, Art Unit 2119
Prosecution Insights