DETAILED ACTION
Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Status of Claims
This action is in reply to the application filed on December 7th, 2023, the response and amendments filed 11/26/2025, and the request for continued examination filed 4/23/2026.
Claims 1 and 19 are currently amended
Claim 18 has been previously amended.
No claims have been added.
No claims have been cancelled.
Claims 1-20 are currently pending and have been examined.
Continued Examination Under 37 CFR 1.114
A request for continued examination under 37 CFR 1.114, including the fee set forth in 37 CFR 1.17(e), was filed in this application after final rejection. Since this application is eligible for continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 4/23/2026 has been entered.
Priority
Receipt is acknowledged of certified copies of papers required by 37 CFR 1.55.
Information Disclosure Statement
The information disclosure statement(s) (IDS(s)) submitted on 12/07/2023 and 01/04/2024 have been received and considered, and the correction of Emanuel 20090019837 to 20090198371 is acknowledged.
Response to Arguments
Applicant’s arguments, see pages 8-12, filed 11/26/2025, with respect to the rejection(s) of independent claim(s) 1 and 19 under 35 USC 103 have been fully considered but are not persuasive regarding the combination of Brasile (WO 2017042677), Schmidt (DE-4429016m), and Simon (US 20170372188). However, the rejection has been withdrawn and a new grounds of rejection has been made in view of Brasile, Simon, and Jeong (US 20110194755) which consolidates the prior art teaching the independent and dependent claims with more clarity. The argument presented on pages 12-14 regarding Gutmann (WO 2013071190) is moot, the rejection being withdrawn in favor of a new grounds of rejection in view of Brasile, Simon, and Jeong.
In response to applicant's arguments against the references individually, one cannot show nonobviousness by attacking references individually where the rejections are based on combinations of references. See In re Keller, 642 F.2d 413, 208 USPQ 871 (CCPA 1981); In re Merck & Co., 800 F.2d 1091, 231 USPQ 375 (Fed. Cir. 1986). The applicant’s arguments against the combination of Brasile (WO 2017042677), Schmidt (DE-4429016m), and Simon (US 20170372188) again amount to a recitation that each piece of prior art in isolation does not teach the complete claim, but only provides a perfunctory dismissal that the rejection is based on a combination of the elements taught in respective pieces of prior art. Applicant acknowledges that Brasile teaches map generation using cameras on a plurality of vehicles, but does not teach upward-directed cameras creating a ceiling feature map or the total combined elements of the claim. While Schmidt is no longer relied upon for rejection, applicant acknowledges that Schmidt teaches the creation (generation) of a ceiling feature map using an upward-directed camera, but does not teach multiple vehicles creating a three-dimensional map, localization on a predefined map, or the total combined elements of the claim. Applicant acknowledges that Simon teaches a pre-existing map, but does not teach generation of a map, and applicant does not address Simon’s teaching of a three-dimensional feature map. The respective elements taught by each prior art element of the new grounds of rejection, their methods of combination, and the motivations for combining them are laid out fully in the following rejections.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis (i.e., changing from AIA to pre-AIA ) for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
The factual inquiries for establishing a background for determining obviousness under 35 U.S.C. 103 are summarized as follows:
1. Determining the scope and contents of the prior art.
2. Ascertaining the differences between the prior art and the claims at issue.
3. Resolving the level of ordinary skill in the pertinent art.
4. Considering objective evidence present in the application indicating obviousness or nonobviousness.
This application currently names joint inventors. In considering patentability of the claims the examiner presumes that the subject matter of the various claims was commonly owned as of the effective filing date of the claimed invention(s) absent any evidence to the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor and effective filing dates of each claim that was not commonly owned as of the effective filing date of the later invention in order for the examiner to consider the applicability of 35 U.S.C. 102(b)(2)(C) for any potential 35 U.S.C. 102(a)(2) prior art against the later invention.
Claim(s) 1, 6-12, 14, and 16-20 are rejected under 35 U.S.C. 103 as being unpatentable over Brasile et al (WO 2017042677, hereinafter “Brasile”) in view of Simon et al (US 20170372188, hereinafter “Simon”), and Jeong et al (US 20110194755, hereinafter “Jeong”).
Regarding Claim 1, Brasile teaches:
Method for person detection for collision avoidance in a location with multiple industrial vehicles, said method comprising the following steps: […] (Brasile Pg 3 lines 7-10 “It is a further object of the invention to provide such a system, which is able to determine if the individual is positioned in a dangerous place because it is used for handling containers by means of vehicles,”)
a. generating or updating a […] map of the location by processing images from an […] camera mounted on at least a first and a second of the industrial vehicles, wherein the […] camera of each of the first and second of the industrial vehicles provide the images for generating or updating the […] feature map; […] (Brasile Pg 4 ¶ 2 “In an exemplary embodiment of the invention, a plurality of industrial vehicles is provided, each industrial vehicle of said plurality being provided with a respective vehicular processing unit, which is arranged to process the data acquired by the video-cameras mounted on the same industrial vehicle for determining the presence of one, or more, individuals within the working area and their spatial position within the working area. Practically, combining all the results obtained processing all the data, a map is obtained of the relative positions of the industrial vehicles with respect to the individual, or the individuals, who are present within the working area,” teaching the generation of a map (a map is obtained) of an area using combined images from cameras on a plurality of industrial vehicles in a space)
[…] onto a two-dimensional floor plan; (Brasile Pg 17 lines 28-30 “generate a geo-referenced map of all the vehicles 10a, 10b and of all the individuals 25a, 25b that are within the working area 200,” Fig 8 showing the geo-referenced map, a floorplan map of working area 200, shown in 2 dimensions)
PNG
media_image1.png
715
514
media_image1.png
Greyscale
c. determining the absolute position of the first industrial vehicle on the floor plan […] (Brasile Pg 3 line 29 – Pg 4 line 2 “unit arranged to instantaneously determine the position of the industrial vehicle within said working area and to generate corresponding position data,”)
[…] d. determining the absolute position of the second industrial vehicle on the floor plan […] (Brasile Pg 3 line 29 – Pg 4 line 2 “unit arranged to instantaneously determine the position of the industrial vehicle within said working area and to generate corresponding position data,”)
[…] e. detecting a person on images from at least one laterally-directed camera mounted on the second industrial vehicle; (Brasile Pg 3 lines 24-27 “a plurality of video-cameras mounted on said industrial vehicle, each video-camera of said plurality being arranged to acquire a sequence of images at a respective angle of view,” shown as items 50 in Fig 2 to be directed laterally as opposed to be vertically)
PNG
media_image2.png
380
610
media_image2.png
Greyscale
f. determining the relative position of said person on the floor plan relative to said second industrial vehicle based on said images; (Brasile Pg 4 lines 3-8 “process said image data […] to determine, through said processing, the presence of an individual and his/her distance from said industrial vehicle;” and Pg 14 lines 22-28 “the processing unit 150 can carry out an additional processing of the images acquired by the video-cameras 50, in order to determine the angle β formed between the optical axis 51 of video-camera 50 and the position of the individual 25. Using known geometric relations, it is, then, possible, to univocally determine the position of individual 25 with respect to vehicle 10,” teaching the calculation of both distance and angle to determine the relative location of the detected individual)
g. determining the absolute position of said person on the floor plan by means of the relative position of said person to the second industrial vehicle and the absolute position of said second industrial vehicle on the floor plan; (Brasile Pg 5 lines 9-13 “said processing unit being, furthermore, arranged to process said position data acquired by said GPS unit for determining the spatial position of said industrial vehicle and, therefore, the spatial position of said individual,” and Pg 15 lines 1-5 “by combining the exact position of the individual 25 with respect to vehicle 10, and the position data obtained by GPS unit 400, it is possible to determine, with a high level of precision, the spatial position of the individual 25,”)
h. determining an alarm contour for the first industrial vehicle (Brasile Pg 17 line 31 – Pg 18 line 11 “According to another aspect of the invention, an emitting device can be provided for emitting an alarm signal, for example visual and/or audio alarm signal, if the detected individual is at a distance from the industrial vehicle 10 less than a predetermined threshold distance d*. It is also provided that, if the detected individual is at a distance less than the threshold distance d* , or a second threshold distance d2* less than d* , a stop procedure can be immediately started del industrial vehicle 10. In particular, d2* can be set between 4 and 6 m, for example 5 m, whilst d* can be set between 5 and 10 m,” teaching a first and second alarm contour as described in ¶ 0058 of the specification: “This alarm contour defines the zone in which presence of a person is relevant for the vehicle, in terms of safety for the person and/or vehicle. This contour can be influenced by a number of factors, such as current speed, max speed, orientation, load, but can also be influenced by environmental factors such as obstacles and objects around it. For instance, where the alarm contour for a vehicle may for instance be set at 10 meter around it in every direction as a standard”)
on said floor plan; and, (Brasile Pg 7 line 25 – Pg 8 line 1 “a verifying step is, furthermore, provided for verifying if the detected individual is positioned in a predetermined dangerous position, and being, furthermore, provided an emitting step for emitting an alarm signal if said spatial position of said detected individual coincides with said predetermined dangerous position,”)
i. providing the absolute position of said person on the floor plan […] (Brasile Pg 19 lines 19-30 “a plurality of industrial vehicles are provided, for example 2 vehicles 10a e 10b, each of which provided with a respective vehicular processing unit 150a and 150b, respectively, arranged to process the data acquired by the video-cameras 50 mounted on the same industrial vehicle. The data processed by each vehicular processing unit 150a and 150b, are sent to the central processing unit 250, which provides, as above disclosed, to supplement these data with those acquired by the fixed video-cameras and to generate a geo-referenced map of all the vehicles 10a, 10b and of all the individuals 25a, 25b that are within the working area 200,”)
[…]wherein an alarm action is triggered for the first industrial vehicle if the absolute position of said person is detected inside the alarm contour of the first industrial vehicle. (Brasile Pg 17 line 31 – Pg 18 line 11 “According to another aspect of the invention, an emitting device can be provided for emitting an alarm signal, for example visual and/or audio alarm signal, if the detected individual is at a distance from the industrial vehicle 10 less than a predetermined threshold distance d*. It is also provided that, if the detected individual is at a distance less than the threshold distance d*, or a second threshold distance d2* less than d* , a stop procedure can be immediately started del industrial vehicle 10. In particular, d2* can be set between 4 and 6 m, for example 5 m, whilst d* can be set between 5 and 10 m.,”)
Brasile does not teach:
[…] three-dimensional feature […]
[…] upwards-directed […]
[…] upwards-directed […]
[…] three-dimensional […]
[…] b. fitting said three-dimensional feature map […]
[…] using images from said upwards directed camera of said first industrial vehicle and the three-dimensional feature map; […]
[…] using images from said upwards directed camera of said second industrial vehicle and the three-dimensional feature map; […]
[…]to the first industrial vehicle, […]
Within the same field of endeavor as Brasile, Jeong teaches:
[…] a. generating or updating a […] feature map of the location by processing images from an upwards-directed camera mounted on at least a first […] industrial vehicles, (Jeong ¶ 0035 lines 1-4 “Specifically, the position estimating unit 142 may create a feature map including feature points extracted from the ceiling image obtained by the image capturing unit 130 and feature point descriptors that identify the respective feature points,” teaching creating a feature map of ceiling features)
wherein the upwards-directed camera of each of the first […] industrial vehicles provide the images for generating or updating the three-dimensional feature map; (Jeong ¶ 0029 “The image capturing unit 130 may capture an external image and convert the image into a digital signal. […] The image capturing unit 130 may obtain a ceiling image by capturing an image in a direction perpendicular to a plane in which the mobile robot 100 travels, for example,” teaching an upwards-directed camera)
[…] b. fitting said […] feature map onto a two-dimensional floor plan; c. determining the absolute position of the first industrial vehicle on the floor plan using images from said upwards directed camera of said first industrial vehicle and the […] feature map; d. determining the absolute position of the second industrial vehicle on the floor plan using images from said upwards directed camera of said second industrial vehicle and the […] feature map; […] (Jeong ¶ 0034 lines 1-8 “Furthermore, the position estimating unit 142 may perform a SLAM algorithm to recognize a position of the mobile robot 100 and simultaneously build up a map. The SLAM algorithm enables a mobile robot to build up a map of surroundings where the mobile robot is located while at the same time tracking its current location based on the built map. The position estimating unit 142 may estimate a current position based on an obstacle map or the ceiling map,” teaching position determine based on the built map, previously described as the feature map, and therefore estimating location on an obstacle map, analogous here to a floorplan, interpreting “absolute position” according to the instant specification ¶ 0033 as being a position in a given coordinate system, i.e. the floor plan, and ¶ 0028 “In detail, a directional angle and a position of the mobile robot 100 in two-dimensional coordinates may be estimated […] The sensor unit 120 may include an infrared, laser, or ultrasonic sensor to obtain obstacle detection information used for building up an obstacle map, noting that additional and/or alternative sensing elements are equally available,” describing the obstacle map as a 2-d floorplan, obtained separately and distinctly from the ceiling feature map.)
Brasile and Jeong are considered analogous because they both relate to industrial vehicle map creation and navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the map creation using cameras on a plurality of industrial vehicles and GPS navigation using a floorplan map of Brasile with the simple addition of Jeong’s upward-directed cameras on an analogous mobile robot that create a feature map of the industrial space including the ceiling on to Brasile’s plurality of vehicles creating a map of the space, thus adding the ceiling feature map into the mapping of Brasile, as well as the substitution of Jeong’s SLAM method of determining vehicle position while mapping for Brasile’s GPS localization. This modification would be made with a reasonable expectation of success as motivated by improving mobile robot performance efficiency and natural movement during execution of tasks by enabling path planning based on recognized ceiling patterns (Jeong ¶ 0044 lines 4-7).
The combination of Brasile and Jeong does not teach:
[…] three-dimensional […]
[…] three-dimensional […]
[…] three-dimensional […]
[…] three-dimensional […]
[…] three-dimensional […]
[…] to the first industrial vehicle, […]
Within the same field of endeavor as Brasile and Jeong, Simon teaches:
[…] three-dimensional [feature map…] three-dimensional [feature map…] three-dimensional [feature map…] three-dimensional feature [feature map…] three-dimensional feature [feature map…] (Simon ¶ 0053 “The environmental-based location tracking device 222 is aware of the absolute position of the industrial vehicle 108 within a dimensionally limited environment, e.g., a mapped portion of a warehouse. By “absolute” position, it is meant that the vehicle position is known relative to a map. The map may be a regional area, e.g., only a portion of an indoor facility such as a warehouse. […] orientation and absolute position are known,” and ¶ 0144 “For instance, in an example implementation, an environmental-based location tracking device on a select industrial vehicle identifies the absolute location of the industrial vehicle within a limited, defined environment, e.g., within a map supported by the environmental-based location tracking system. […] In practice, the locations could be expressed in more defined terms, such as latitude and longitude, X, Y, and Z coordinates, or any other coordinates. In this regard, the location of the badge can be expressed in absolute terms, e.g., coordinates on the map,” and ¶ 0146 “For instance, the server 112 can load a map of a defined environment, where the map has features that characterize the physical layout of the defined environment. As an example, loading a map of a defined environment can be implemented by loading a map that has defined aisles that indicate where an electronic badge 126 is allowed to navigate, and restricted areas where the electronic badge 126 cannot navigate through,” teaching the use of a digital map with 3-dimensional coordinates of the warehouse environment consisting of features in the environment)
[…] to the first industrial vehicle, […] (Simon ¶ 0206 lines 14-24 “For instance, the second industrial vehicle 108B can receive an information message, e.g., from […] the first industrial vehicle 108 via the badge communicator 224, […] In response to processing the information message, a processor on the second industrial vehicle 108B generates on a display, a map of the work environment, first indicia representing the position of the first industrial vehicle and second indicia representing the position of the electronic badge detected by the first industrial vehicle,” teaching one industrial vehicle sending location information of an identified person (electronic badge detected) to another industrial vehicle)
Brasile, Jeong, and Simon are all considered analogous because they all relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the planar floorplan map shown in 2-dimensions of Brasile and the generally presented ceiling feature map of Jeong which is neither specified nor limited in terms of dimension by adding the functionality of the digital map with 3-dimensional coordinates and warehouse features of Simon. This modification would be made with a reasonable expectation of success as motivated by increased awareness of environmental constraints on the robots and people (Simon ¶ 0069), and furthermore according to MPEP 2143(I)(E), it would be obvious to one skilled in the art to try combining the estimation of position on an two-dimensional obstacle map (analogous to a floor plan) by mapping using features of a ceiling map of unspecified dimensionality with the 3d map of Simon, as transforming from a 2d light map or a 3d feature map into a 2d map constitutes a limited number of identified, predictable solutions. It would also have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Brasile’s sending of processed position data to central processing by adding Simon’s message including first indicia representing the position of one industrial vehicle and second indicia representing the position of the electronic badge detected by the one industrial vehicle directly from the one industrial vehicle to another industrial vehicle. This modification would be made with a reasonable expectation of success as motivated by increased awareness and reduced computation and transmission time, as would be obvious to one of ordinary skill in the art.
Regarding Claim 6, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile does not teach:
wherein the step of generating or updating a three-dimensional feature map of the location is carried out using a simultaneous location and mapping (SLAM) approach.
Within the same field of endeavor as Brasile, Jeong teaches:
wherein the step of generating or updating a three-dimensional feature map of the location is carried out using a simultaneous location and mapping (SLAM) approach. ((Jeong ¶ 0034 lines 1-8 “Furthermore, the position estimating unit 142 may perform a SLAM algorithm to recognize a position of the mobile robot 100 and simultaneously build up a map. The SLAM algorithm enables a mobile robot to build up a map of surroundings where the mobile robot is located while at the same time tracking its current location based on the built map. The position estimating unit 142 may estimate a current position based on an obstacle map or the ceiling map,” teaching generating the map while determining location using simultaneous localization and mapping (SLAM) applied to the 3-dimensional map of Simon)
Brasile and Jeong are considered analogous because they both relate to industrial vehicle map creation and navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the map creation using cameras on a plurality of industrial vehicles and GPS navigation using a floorplan map of Brasile with the simple addition of Jeong’s upward-directed cameras on an analogous mobile robot that create a feature map of the industrial space including the ceiling on to Brasile’s plurality of vehicles creating a map of the space, thus adding the ceiling feature map into the mapping of Brasile, as well as the substitution of Jeong’s SLAM method of determining vehicle position while mapping for Brasile’s GPS localization. This modification would be made with a reasonable expectation of success as motivated by improving mobile robot performance efficiency and natural movement during execution of tasks by enabling path planning based on recognized ceiling patterns (Jeong ¶ 0044 lines 4-7).
Regarding Claim 7, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile does not teach:
wherein the alarm contour of the industrial vehicle is calculated based on at least one of the speed,
acceleration,
mass,
volume
and direction of travel of the industrial vehicle.
Within the same field of endeavor as Brasile, Simon teaches:
wherein the alarm contour of the industrial vehicle is calculated based on at least one of (Simon ¶ 0078 “An “awareness zone” is a zone, such as an arbitrary, virtual zone that is contained within and can extend up to, but not beyond a corresponding detection zone. Since an awareness zone is virtual, a given awareness zone can take any desired shape only constrained by the corresponding detection zone. According to aspects of the present disclosure, an awareness zone for detecting an electronic badge 126 by the badge communicator 224 in proximity of the industrial vehicle 108 can be dynamically altered based upon predetermined criteria. The modification of at least one awareness zone is referred to herein as zone ranging,”)
the speed, (Simon ¶ 0080 lines 1-3 “In an example implementation, the size of the awareness zone dynamically changes based upon vehicle speed,”)
acceleration, (Simon ¶ 0092 lines 6-10 “This allows the information linking device 202, e.g., via information received from the server 112, to take standard vehicle performance, such as acceleration/deceleration curves […] of the vehicle into consideration in defining the size of the awareness zone,”)
mass, volume (Simon ¶ 0090 lines 16-17 “As an example, the higher and/or heavier the load, the larger the awareness zone,” height and heaviness analogous to volume and mass respectively)
and direction of travel of the industrial vehicle. (Simon ¶ 0089 lines 5-6 “zone ranging may be based upon drive/travel direction,”)
Brasile and Simon are both considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the alarm predetermined threshold distances of Brasile with the simple substitution of Simon’s dynamically updating awareness zone modified by zone ranging based on speed, acceleration curves, height and weight of load, or travel direction. This modification would be made with a reasonable expectation of success as motivated by improving safety and efficiency by scaling the awareness zone appropriately for a given situation based on risk factors.
Regarding Claim 8, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile further teaches:
wherein the alarm action comprises reducing speed […] (Brasile Pg 8 lines 12-15 “if the detected individual is distance from the industrial vehicle less than a second predetermined threshold distance d2*<d*, a stop step can be provided to completely stop the industrial vehicle,” teaching an alarm action of reducing speed to a stop)
Brasile does not teach:
[…] to a maximum of 5 km/h.
Within the same field of endeavor as Brasile, Simon teaches:
[…] to a maximum of 5 km/h. (Simon ¶ 0094 “Referring to FIG. 5, the badge communicator 224, information linking device 202, and industrial vehicle 108 can cooperate to generate multiple simultaneous zones. This allows, for example, the utilization of a […] an action zone (where some control function happens on the industrial vehicle 108—e.g., set points are changed, top speed is limited, etc.),” teaching a maximum speed limitation as a response to a person being identified within a zone, and ¶ 0102 “As a working example of a top speed reduction application, a vehicle top speed is dynamically altered by the system based upon whether an electronic badge 126 is detected in a particular zone,” teaching selection of a maximum speed based on the situation, analogous to a particular set top speed such as 5 km/h)
Brasile and Simon are both considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the alarm action to reduce speed to a stop of Brasile with the simple substitution of Simon’s top speed limit if within an action zone. This modification would be made with a reasonable expectation of success as motivated by improving efficiency by scaling the vehicle response and reducing unnecessary stops.
Regarding Claim 9, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile does not teach:
wherein the floor plan comprises at least one designated pedestrian zones,
and wherein the alarm action is not triggered if the absolute position of said person is detected inside the alarm contour of the first industrial vehicle and said absolute position is in one of the at least one designated pedestrian zones.
Within the same field of endeavor as Brasile, Simon teaches:
wherein the floor plan comprises at least one designated pedestrian zones, (Simon ¶ 0189 lines 1-4 “Referring to FIG. 15, according to further aspects of the present disclosure, the system can use geo-based features to create “exclude zones” or exceptions from a generated awareness zone,” and ¶ 0191 lines 9-14 “The information linking device 202 can receive from the server 112 via the environmental-based location tracking 222, or otherwise determine information indicating that the electronic badge 1508 is in an exclude zone, e.g., a safe zone behind a barricade 1514. The barricade 1514 can be noted by coordinates on a CAD map or other format,”)
and wherein the alarm action is not triggered if the absolute position of said person is detected inside the alarm contour of the first industrial vehicle and said absolute position is in one of the at least one designated pedestrian zones. (Simon ¶ 0191 lines 15-18 “As such, the information linking device 202 warns the vehicle operator of only electronic badge 1510 despite three electronic badges being in the awareness zone,” illustrated in Fig 15 to exclude 1508 in the exclude zone)
PNG
media_image3.png
568
491
media_image3.png
Greyscale
Brasile and Simon are both considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the alarm predetermined threshold distances of Brasile with the addition of Simon’s exclude zone where pedestrians are excluded from a warning zone. This modification would be made with a reasonable expectation of success as motivated by reducing unnecessary alarm actions for pedestrians within designated pedestrian areas.
Regarding Claim 10, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile further teaches:
wherein the method further comprises a step of: each industrial vehicle broadcasting the absolute position of any person it detects. (Brasile Pg 17 lines 24-30 “The data processed by each vehicular processing unit 150a and 150b, are sent to the central processing unit 250, which provides, as above disclosed, to supplement these data with those acquired by the fixed video-cameras and to generate a geo-referenced map of all the vehicles 10a, 10b and of all the individuals 25a, 25b that are within the working area 200,”)
Regarding Claim 11, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile further teaches:
wherein one or more fixed camera kits are positioned in the location, (Brasile Pg 5 lines 29 – Pg 6 line 1 “a plurality of fixed video-cameras is, furthermore, provided adapted to acquire a plurality of additional images of said working area,”)
configured for detecting a person on images from said stationary camera kit and determining the absolute position of said detected person on the floor plan, (Brasile Pg 6 lines 1-6 “In particular, the processing unit comprises a central processing unit arranged to process the plurality of additional images and to supplement the results of data processing with the results of said, or each, vehicular processing unit, in such a way to generate a geo-referenced map of the industrial vehicles and of the individuals that are present within the working area,”)
wherein the stationary camera kit is configured for broadcasting said absolute location, (Brasile Pg 8 lines 24-27 “In an exemplary embodiment of the invention, also the images acquired by the fixed video-cameras, are displayed on a display, which is placed inside the cockpit of the industrial vehicle,”)
and wherein the step of triggering the alarm action takes into account the absolute position received from the fixed camera kits. (Brasile Pg 17 lines 24 – Pg 18 line 4 “The data processed by each vehicular processing unit 150a and 150b, are sent to the central processing unit 250, which provides, as above disclosed, to supplement these data with those acquired by the fixed video-cameras and to generate a geo-referenced map of all the vehicles 10a, 10b and of all the individuals 25a, 25b that are within the working area 200. According to another aspect of the invention, an emitting device can be provided for emitting an alarm signal, for example visual and/or audio alarm signal, if the detected individual is at a distance from the industrial vehicle 10 less than a predetermined threshold distance d*.,”)
Regarding Claim 12, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile does not teach:
wherein the three-dimensional feature map comprises ceiling lights as features,
and optionally skylights,
racking,
gates,
and/or windows.
Within the same field of endeavor as Brasile, Schmidt teaches:
wherein the three-dimensional feature map comprises ceiling lights as features, (Jeong ¶ 0031 lines 1-10 “The feature point extracting unit 141 may extract a feature point from the ceiling image obtained by the image capturing unit 130. The feature point refers to a point such as an edge or a corner which can specify a shape of an object. The feature point may be referred to as a landmark based on which a feature point map can be built. […] For example, a line or point may be extracted as the feature point from a circular or quadrangular outline of an object such as a light in the ceiling image,” as applied to the 3-dimensional map of Simon)
Brasile and Jeong are considered analogous because they both relate to industrial vehicle map creation and navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the map creation using cameras on a plurality of industrial vehicles and GPS navigation using a floorplan map of Brasile with the simple addition of Jeong’s upward-directed cameras on an analogous mobile robot that create a feature map of the industrial space including the ceiling and furthermore lights on the ceiling on to Brasile’s plurality of vehicles creating a map of the space, thus adding the ceiling feature map into the mapping of Brasile, as well as the substitution of Jeong’s SLAM method of determining vehicle position while mapping for Brasile’s GPS localization. This modification would be made with a reasonable expectation of success as motivated by improving mobile robot performance efficiency and natural movement during execution of tasks by enabling path planning based on recognized ceiling patterns (Jeong ¶ 0044 lines 4-7).While not relied upon for this rejection, Simon teaches the use of racking as a map feature within 3D maps in ¶ 0069.
Regarding Claim 14, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile does not teach:
wherein the three-dimensional feature map comprises racking as features.
Within the same field of endeavor as Brasile, Simon teaches:
wherein the three-dimensional feature map comprises racking as features. (Simon Pg 4 ¶ 4 lines 1-2 “However, the environmental-based location and tracking device 222 can detect the absolute position of the industrial vehicle 108 and is thus aware of static environmental constraints, e.g., via a map that is limited to a pre-mapped section of a warehouse. Here, “static environmental constraints” includes features such as […] rack location,”)
Brasile and Simon are considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the floorplan map GPS navigation of Brasile and 3D digital environment feature map navigation of Simon by including the mapped rack locations of Simon. This modification would be made with a reasonable expectation of success as motivated by increased awareness of environmental constraints on the robots and people (Simon ¶ 0069).
Regarding Claim 16, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile further teaches:
[…] camera mounted on all of the industrial vehicles. (Brasile Pg 17 lines 19-24 “a plurality of industrial vehicles are provided, for example 2 vehicles 10a e 10b, each of which provided with a respective vehicular processing unit 150a and 150b, respectively, arranged to process the data acquired by the video-cameras 50 mounted on the same industrial vehicle.”)
Brasile does not teach:
wherein the three-dimensional feature map of the location is generated and updated using an upwards-directed […]
Within the same field of endeavor as Brasile, Jeong teaches:
[…] feature map of the location is generated and updated using an upwards-directed […] (Jeong ¶ 0035 lines 1-4 “Specifically, the position estimating unit 142 may create a feature map including feature points extracted from the ceiling image obtained by the image capturing unit 130 and feature point descriptors that identify the respective feature points,” teaching creating a feature map of ceiling features and ¶ 0029 “The image capturing unit 130 may capture an external image and convert the image into a digital signal. […] The image capturing unit 130 may obtain a ceiling image by capturing an image in a direction perpendicular to a plane in which the mobile robot 100 travels, for example,” teaching an upwards-directed camera)
Brasile and Jeong are considered analogous because they both relate to industrial vehicle map creation and navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the map creation using cameras on a plurality of industrial vehicles and GPS navigation using a floorplan map of Brasile with the simple addition of Jeong’s upward-directed cameras on an analogous mobile robot that create a feature map of the industrial space including the ceiling on to Brasile’s plurality of vehicles creating a map of the space, thus adding the ceiling feature map into the mapping of Brasile, as well as the substitution of Jeong’s SLAM method of determining vehicle position while mapping for Brasile’s GPS localization. This modification would be made with a reasonable expectation of success as motivated by improving mobile robot performance efficiency and natural movement during execution of tasks by enabling path planning based on recognized ceiling patterns (Jeong ¶ 0044 lines 4-7).
The combination of Brasile and Jeong does not teach:
wherein the three-dimensional feature […]
Within the same field of endeavor as Brasile and Jeong, Simon teaches:
wherein the three-dimensional feature (Simon ¶ 0053 “The environmental-based location tracking device 222 is aware of the absolute position of the industrial vehicle 108 within a dimensionally limited environment, e.g., a mapped portion of a warehouse. By “absolute” position, it is meant that the vehicle position is known relative to a map. The map may be a regional area, e.g., only a portion of an indoor facility such as a warehouse. […] orientation and absolute position are known,” and ¶ 0144 “For instance, in an example implementation, an environmental-based location tracking device on a select industrial vehicle identifies the absolute location of the industrial vehicle within a limited, defined environment, e.g., within a map supported by the environmental-based location tracking system. […] In practice, the locations could be expressed in more defined terms, such as latitude and longitude, X, Y, and Z coordinates, or any other coordinates. In this regard, the location of the badge can be expressed in absolute terms, e.g., coordinates on the map,” and ¶ 0146 “For instance, the server 112 can load a map of a defined environment, where the map has features that characterize the physical layout of the defined environment. As an example, loading a map of a defined environment can be implemented by loading a map that has defined aisles that indicate where an electronic badge 126 is allowed to navigate, and restricted areas where the electronic badge 126 cannot navigate through,” teaching the use of a digital map with 3-dimensional coordinates of the warehouse environment consisting of features in the environment)
Brasile, Jeong, and Simon are all considered analogous because they all relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the floorplan map and plurality of vehicles with cameras of Brasile and ceiling map generation of Schmidt by adding the functionality of the digital map with 3-dimensional coordinates and warehouse features of Simon. This modification would be made with a reasonable expectation of success as motivated by increased awareness of environmental constraints on the robots and people (Simon ¶ 0069).
Regarding Claim 17, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile does not teach:
wherein the location is one or more warehouses.
Within the same field of endeavor as Brasile, Simon teaches:
wherein the location is one or more warehouses. (Simon ¶ 0026 lines 1-4 “According to various aspects of the present disclosure, systems and computer-implemented processes provide communication between electronic badges operating in a constrained environment such as a warehouse,”)
Brasile and Simon are considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the navigation industrial vehicles within a port terminal (Brasile Pg 1 lines 6-7) of Brasile with the simple substitution of Simon’s warehouse environment. This modification would be made with a reasonable expectation of success as motivated by applying a known technique to a known device (method, or product) ready for improvement to yield predictable results (MPEP(I)(D)).
Regarding Claim 18, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile further teaches:
wherein the industrial vehicle is a mobile material handling unit. (Brasile Pg 3 lines 2-6 “It is, therefore, an object of the present invention to provide a system for determining the position of an individual within a determined working area, in particular a port area, where an industrial vehicle operates for handling containers,” emphasis added)
Regarding Claim 19, Brasile teaches:
System for person detection in collision avoidance for industrial vehicles in a location, (Brasile Pg 3 lines 7-10 “It is a further object of the invention to provide such a system, which is able to determine if the individual is positioned in a dangerous place because it is used for handling containers by means of vehicles,”)
wherein the system comprises a plurality of vehicle kits provided on each of the industrial vehicles, (Brasile Pg 19 lines 19-30 “a plurality of industrial vehicles are provided, for example 2 vehicles 10a e 10b, each of which provided with a respective vehicular processing unit 150a and 150b, respectively, arranged to process the data acquired by the video-cameras 50 mounted on the same industrial vehicle. The data processed by each vehicular processing unit 150a and 150b, are sent to the central processing unit 250, which provides, as above disclosed, to supplement these data with those acquired by the fixed video-cameras and to generate a geo-referenced map of all the vehicles 10a, 10b and of all the individuals 25a, 25b that are within the working area 200,”)
each kit comprising: […] b. at least one person detection camera mounted on the industrial vehicle, directed laterally with respect to the vehicle; (Brasile Pg 3 lines 24-27 “a plurality of video-cameras mounted on said industrial vehicle, each video-camera of said plurality being arranged to acquire a sequence of images at a respective angle of view,” shown as items 50 in Fig 2 to be directed laterally as opposed to be vertically)
c. a processing unit configured for:
i. processing images from the first camera, wherein images provided by the first cameras of two or more vehicle kits contribute to generating or updating a […] map;
ii. determining an absolute position of the industrial vehicle (Brasile Pg 3 line 29 – Pg 4 line 2 “unit arranged to instantaneously determine the position of the industrial vehicle within said working area and to generate corresponding position data,” and Pg 5 lines 8-15 “In particular, the processing unit comprises at least a vehicular processing unit mounted on the vehicle, said vehicular processing unit being arranged to process said data acquired by said video-cameras, which are mounted on said industrial vehicle, for determining the presence, or not, of an individual within the working area, and for determining the distance between the individual and the 15 industrial vehicle,”)
on a […] floor plan […] feature map of the location; (Brasile Pg 17 lines 28-30 “generate a geo-referenced map of all the vehicles 10a, 10b and of all the individuals 25a, 25b that are within the working area 200,” Fig 8 showing the geo-referenced map, a floorplan map of working area 200)
iii. detecting a person on images from the person detection camera (Brasile Pg 4 lines 3-7 “a processing unit arranged to process said image data acquired by said plurality of video-cameras by applying at least a predetermined object recognition algorithm, said processing unit arranged to determine, through said processing, the presence of an individual,”)
and determining the relative position of said person relative to said industrial vehicle based on said images on the floor plan; (Brasile Pg 4 lines 3-8 “process said image data […] to determine, through said processing, the presence of an individual and his/her distance from said industrial vehicle;” and Pg 14 lines 22-28 “the processing unit 150 can carry out an additional processing of the images acquired by the video-cameras 50, in order to determine the angle β formed between the optical axis 51 of video-camera 50 and the position of the individual 25. Using known geometric relations, it is, then, possible, to univocally determine the position of individual 25 with respect to vehicle 10,” teaching the calculation of both distance and angle to determine the relative location of the detected individual)
iv. determining an absolute position of said person on the floor plan by means of the relative position of said person to the industrial vehicle and the absolute position of said industrial vehicle on the floor plan; (Brasile Pg 5 lines 9-13 “said processing unit being, furthermore, arranged to process said position data acquired by said GPS unit for determining the spatial position of said industrial vehicle and, therefore, the spatial position of said individual,” and Pg 15 lines 1-5 “by combining the exact position of the individual 25 with respect to vehicle 10, and the position data obtained by GPS unit 400, it is possible to determine, with a high level of precision, the spatial position of the individual 25,”)
v. determining an alarm contour for the industrial vehicle (Brasile Pg 17 line 31 – Pg 18 line 11 “According to another aspect of the invention, an emitting device can be provided for emitting an alarm signal, for example visual and/or audio alarm signal, if the detected individual is at a distance from the industrial vehicle 10 less than a predetermined threshold distance d*. It is also provided that, if the detected individual is at a distance less than the threshold distance d* , or a second threshold distance d2* less than d* , a stop procedure can be immediately started del industrial vehicle 10. In particular, d2* can be set between 4 and 6 m, for example 5 m, whilst d* can be set between 5 and 10 m,” teaching a first and second alarm contour as described in ¶ 0058 of the specification: “This alarm contour defines the zone in which presence of a person is relevant for the vehicle, in terms of safety for the person and/or vehicle. This contour can be influenced by a number of factors, such as current speed, max speed, orientation, load, but can also be influenced by environmental factors such as obstacles and objects around it. For instance, where the alarm contour for a vehicle may for instance be set at 10 meter around it in every direction as a standard”)
on said floor plan; […] (Brasile Pg 7 line 25 – Pg 8 line 1 “a verifying step is, furthermore, provided for verifying if the detected individual is positioned in a predetermined dangerous position, and being, furthermore, provided an emitting step for emitting an alarm signal if said spatial position of said detected individual coincides with said predetermined dangerous position,”)
[…] configured for broadcasting the determined absolute positions, […] (Brasile Pg 19 lines 19-30 “a plurality of industrial vehicles are provided, for example 2 vehicles 10a e 10b, each of which provided with a respective vehicular processing unit 150a and 150b, respectively, arranged to process the data acquired by the video-cameras 50 mounted on the same industrial vehicle. The data processed by each vehicular processing unit 150a and 150b, are sent to the central processing unit 250, which provides, as above disclosed, to supplement these data with those acquired by the fixed video-cameras and to generate a geo-referenced map of all the vehicles 10a, 10b and of all the individuals 25a, 25b that are within the working area 200,” emphasis added)
[…] wherein said processing unit is further configured for executing an alarm action if the absolute position of said person is detected inside the alarm contour of the industrial vehicle. (Brasile Pg 17 line 31 – Pg 18 line 11 “According to another aspect of the invention, an emitting device can be provided for emitting an alarm signal, for example visual and/or audio alarm signal, if the detected individual is at a distance from the industrial vehicle 10 less than a predetermined threshold distance d*. It is also provided that, if the detected individual is at a distance less than the threshold distance d* , or a second threshold distance d2* less than d* , a stop procedure can be immediately started del industrial vehicle 10. In particular, d2* can be set between 4 and 6 m, for example 5 m, whilst d* can be set between 5 and 10 m.,”)
Brasile does not teach:
[…] a. a first camera mounted on the industrial vehicle, directed upwards with respect to the vehicle; […]
[…] three-dimensional feature […]
[…] predefined […]
[…] using images from said first camera of said industrial vehicle and the three-dimensional […]
Brasile suggests but does not explicitly disclose […] d. a wireless communication unit, […]
[…] and for receiving broadcasted determined absolute positions from other communication units; […]
Within the same field of endeavor as Brasile, Jeong teaches:
[…] a. a first camera mounted on the industrial vehicle, directed upwards with respect to the vehicle; […] (Jeong ¶ 0029 “The image capturing unit 130 may capture an external image and convert the image into a digital signal. […] The image capturing unit 130 may obtain a ceiling image by capturing an image in a direction perpendicular to a plane in which the mobile robot 100 travels, for example,” teaching an upwards-directed camera)
[…] i. processing images from the first camera, wherein images provided by the first cameras of […] vehicle kits contribute to generating or updating a […] feature map; (Jeong ¶ 0035 lines 1-4 “Specifically, the position estimating unit 142 may create a feature map including feature points extracted from the ceiling image obtained by the image capturing unit 130 and feature point descriptors that identify the respective feature points,” teaching creating a feature map of ceiling features)
ii. determining an absolute position of the industrial vehicle on a predefined floor plan using images from said first camera of said industrial vehicle and […] feature map of the location; […] (Jeong ¶ 0034 lines 1-8 “Furthermore, the position estimating unit 142 may perform a SLAM algorithm to recognize a position of the mobile robot 100 and simultaneously build up a map. The SLAM algorithm enables a mobile robot to build up a map of surroundings where the mobile robot is located while at the same time tracking its current location based on the built map. The position estimating unit 142 may estimate a current position based on an obstacle map or the ceiling map,” teaching position determine based on the built map, previously described as the feature map, and therefore estimating location on an obstacle map, analogous here to a floorplan, interpreting “absolute position” according to the instant specification ¶ 0033 as being a position in a given coordinate system, i.e. the floor plan, and ¶ 0028 “In detail, a directional angle and a position of the mobile robot 100 in two-dimensional coordinates may be estimated […] The sensor unit 120 may include an infrared, laser, or ultrasonic sensor to obtain obstacle detection information used for building up an obstacle map, noting that additional and/or alternative sensing elements are equally available,” describing the obstacle map as a 2-d floorplan, obtained separately and distinctly from the ceiling feature map.)
Brasile and Jeong are considered analogous because they both relate to industrial vehicle map creation and navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the map creation using cameras on a plurality of industrial vehicles and GPS navigation using a floorplan map of Brasile with the simple addition of Jeong’s upward-directed cameras on an analogous mobile robot that create a feature map of the industrial space including the ceiling on to Brasile’s plurality of vehicles creating a map of the space, thus adding the ceiling feature map into the mapping of Brasile, as well as the substitution of Jeong’s SLAM method of determining vehicle position while mapping for Brasile’s GPS localization. This modification would be made with a reasonable expectation of success as motivated by improving mobile robot performance efficiency and natural movement during execution of tasks by enabling path planning based on recognized ceiling patterns (Jeong ¶ 0044 lines 4-7).
The combination of Brasile and Jeong does not teach:
Brasile does not teach:
[…] three-dimensional […]
[…] three-dimensional […]
[…] predefined […]
Brasile suggests but does not explicitly disclose […] d. a wireless communication unit, […]
[…] and for receiving broadcasted determined absolute positions from other communication units; […]
Within the same field of endeavor as Brasile and Jeong, Simon teaches:
[…] three-dimensional […] (Simon ¶ 0053 “The environmental-based location tracking device 222 is aware of the absolute position of the industrial vehicle 108 within a dimensionally limited environment, e.g., a mapped portion of a warehouse. By “absolute” position, it is meant that the vehicle position is known relative to a map. The map may be a regional area, e.g., only a portion of an indoor facility such as a warehouse. […] orientation and absolute position are known,” and ¶ 0144 “For instance, in an example implementation, an environmental-based location tracking device on a select industrial vehicle identifies the absolute location of the industrial vehicle within a limited, defined environment, e.g., within a map supported by the environmental-based location tracking system. […] In practice, the locations could be expressed in more defined terms, such as latitude and longitude, X, Y, and Z coordinates, or any other coordinates. In this regard, the location of the badge can be expressed in absolute terms, e.g., coordinates on the map,” and ¶ 0146 “For instance, the server 112 can load a map of a defined environment, where the map has features that characterize the physical layout of the defined environment. As an example, loading a map of a defined environment can be implemented by loading a map that has defined aisles that indicate where an electronic badge 126 is allowed to navigate, and restricted areas where the electronic badge 126 cannot navigate through,” teaching the use of a digital map with 3-dimensional coordinates of the warehouse environment consisting of features in the environment)
[…]position of the industrial vehicle on a predefined […] (Simon ¶ 0146 lines 1-7 “For instance, the server 112 can load a map of a defined environment, where the map has features that characterize the physical layout of the defined environment. As an example, loading a map of a defined environment can be implemented by loading a map that has defined aisles that indicate where an electronic badge 126 is allowed to navigate,” teaching the navigable environment map (analogous to Jeong’s obstacle map) being a pre-defined, loaded map)
[…] d. a wireless communication unit, […] (Simon ¶ 0035 lines 14-24 “Still further, a processing device 102 is provided on one or more industrial vehicles 108 such as a forklift truck, reach truck, stock picker, automated guided vehicle, turret truck, tow tractor, rider pallet truck, walkie stacker truck, etc. In the example configuration illustrated, the industrial vehicles 108 wirelessly communicate […] the industrial vehicles 108 can be equipped with Wi-Fi, cellular or other suitable technology that allows the processing device 102 on the industrial vehicle 108 to communicate directly with a remote device (e.g., over the networks 104),”)
[…] and for receiving broadcasted determined absolute positions from other communication units; […] (Simon ¶ 0206 lines 14-24 “For instance, the second industrial vehicle 108B can receive an information message, e.g., from […] the first industrial vehicle 108 via the badge communicator 224, […] In response to processing the information message, a processor on the second industrial vehicle 108B generates on a display, a map of the work environment, first indicia representing the position of the first industrial vehicle and second indicia representing the position of the electronic badge detected by the first industrial vehicle,” teaching one industrial vehicle sending location information of an identified person (electronic badge detected) to another industrial vehicle)
Brasile, Jeong, and Simon are all considered analogous because they all relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the planar floorplan map shown in 2-dimensions of Brasile and the generally presented ceiling feature map of Jeong which is neither specified nor limited in terms of dimension by adding the functionality of the digital map with 3-dimensional coordinates and warehouse features of Simon. This modification would be made with a reasonable expectation of success as motivated by increased awareness of environmental constraints on the robots and people (Simon ¶ 0069), and furthermore according to MPEP 2143(I)(E), it would be obvious to one skilled in the art to try combining the estimation of position on an two-dimensional obstacle map (analogous to a floor plan) by mapping using features of a ceiling map of unspecified dimensionality with the 3d map of Simon, as transforming from a 2d light map or a 3d feature map into a 2d map constitutes a limited number of identified, predictable solutions. It would also have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the Brasile’s sending of processed position data to central processing by adding Simon’s message including first indicia representing the position of one industrial vehicle and second indicia representing the position of the electronic badge detected by the one industrial vehicle directly from the one industrial vehicle to another industrial vehicle. This modification would be made with a reasonable expectation of success as motivated by increased awareness and reduced computation and transmission time, as would be obvious to one of ordinary skill in the art.
Regarding Claim 20, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 19 as described above. Brasile further teaches:
wherein the system comprises one or more stationary camera kits, said stationary camera kits comprising a stationary camera, […] (Brasile Pg 5 lines 29 – Pg 6 line 1 “a plurality of fixed video-cameras is, furthermore, provided adapted to acquire a plurality of additional images of said working area,”)
[…] and a processing unit for detecting a person on images from said stationary camera and determining the absolute position of said detected person on the floor plan, (Brasile Pg 6 lines 1-6 “In particular, the processing unit comprises a central processing unit arranged to process the plurality of additional images and to supplement the results of data processing with the results of said, or each, vehicular processing unit, in such a way to generate a geo-referenced map of the industrial vehicles and of the individuals that are present within the working area,”)
wherein the wireless communication unit is configured for broadcasting said absolute location, (Brasile Pg 8 lines 24-27 “In an exemplary embodiment of the invention, also the images acquired by the fixed video-cameras, are displayed on a display, which is placed inside the cockpit of the industrial vehicle,”)
and wherein the processing unit of the vehicle kits takes into account the absolute location received from the stationary camera kits for executing the alarm action. (Brasile Pg 17 lines 24 – Pg 18 line 4 “The data processed by each vehicular processing unit 150a and 150b, are sent to the central processing unit 250, which provides, as above disclosed, to supplement these data with those acquired by the fixed video-cameras and to generate a geo-referenced map of all the vehicles 10a, 10b and of all the individuals 25a, 25b that are within the working area 200. According to another aspect of the invention, an emitting device can be provided for emitting an alarm signal, for example visual and/or audio alarm signal, if the detected individual is at a distance from the industrial vehicle 10 less than a predetermined threshold distance d*.,”)
Brasile does not teach:
[…] a wireless communication unit […]
Within the same field of endeavor as Brasile, Simon teaches:
[…] a wireless communication unit […] (Simon ¶ 0117 “It would also be possible to receive this information from a badge communicator 224 that is not mounted on an industrial vehicle. This could be a stationary badge communicator 224, for example near a charging station or a door. Still further, the process may send the electronic badge identifier to the server via an information linking device on the industrial vehicle, where the information linking device communicates with the server over a wireless communication link that is different from the wireless communication link between the electronic badge and corresponding badge communicator. The server receives the badge identifier, and responds to the industrial vehicle with the appropriate information,” teaching a badge communicator, analogous to the stationary cameras in that it is a static emplacement that tracks positions of individuals, which is statically mounted and not mounted to an industrial vehicle, with a wireless communication unit which broadcasts the badge identifier information)
Brasile and Simon are both considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the fixed video-cameras and central processing unit of Brasile by adding the wireless communication of Simon’s electronic badge identifier not mounted to an industrial vehicle. This modification would be made with a reasonable expectation of success as motivated by increased awareness of environmental constraints on the robots and people (Simon ¶ 0069).
Claim(s) 2-4 is/are rejected under 35 U.S.C. 103 as being unpatentable over Brasile in view of Jeong and Simon and further in view of Son et al (KR 20200048918, hereinafter “Son,” all citations and excerpts taken from the attached machine translation).
Regarding Claim 2, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile does not teach:
wherein the step of determining the absolute position of the first and/or second industrial vehicle is carried out by means of particle filter localization technique.
Within the same field of endeavor as Brasile, Son teaches:
wherein the step of determining the absolute position of the first and/or second industrial vehicle is carried out by means of particle filter localization technique. (Son Pg 9 ¶ 7 lines 8-10 “the positioning is continuously performed using the improved particle filter localization method according to an embodiment of the present invention, and the result is used to locate on the digital map. It will be possible to calibrate the absolute position of the device,”)
Brasile, Jeong, Simon, and Son are all considered analogous because they all relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the floorplan map GPS navigation of Brasile and ceiling map navigation of Jeong, and 3D digital environment feature map navigation of Simon by adding the particle filter localization Son. This modification would be made with a reasonable expectation of success as motivated by efficient and low-cost computation (Son Pg 7 ¶ 1).
Regarding Claim 3, the combination of Brasile, Jeong, Simon, and Son teaches the limitations of Claim 2 as described above. Brasile does not teach:
wherein the absolute position of the industrial vehicle is determined by comparison of an expected feature image
for each particle in the particle filter localization technique based
on the three-dimensional feature map
with the image from the upwards directed camera.
Within the same field of endeavor as Brasile, Jeong and Simon teach the following as described above in claim 1 with appropriate motivation:
the three-dimensional feature map (Simon ¶ 0053, ¶ 0144, and ¶ 0146)
with the image from the upwards directed camera. (Jeong ¶ 0029)
Within the same field of endeavor as Brasile, Son teaches:
wherein the absolute position of the industrial vehicle is determined by comparison of an expected feature image for each particle in the particle filter localization technique based on (Son Pg 7 ¶ 3 “In the process of moving the particles, the spatial probability that the particles will be present is analyzed (S122), and as a result, when it is determined that particles are concentrated at a specific location on the digital map and particle location convergence is achieved (S124), the location is digital map of the positioning device. It is the gist of particle filter localization that is considered to be the actual location of the image (S126),” teaching obtaining the actual location (absolute position) by comparison (particle location convergence) of an expected feature image (digital map) for reach particle (particles are concentrated at a specific location on the digital map))
Brasile, Jeong, Simon, and Son are both considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the floorplan map GPS navigation of Brasile and ceiling map navigation of Jeong, and 3D digital environment feature map navigation of Simon by adding the particle filter localization Son. This modification would be made with a reasonable expectation of success as motivated by efficient and low-cost computation (Son Pg 7 ¶ 1).
Regarding Claim 4, the combination of Brasile, Jeong, Simon, and Son teaches the limitations of Claim 3 as described above. Brasile does not teach:
wherein the image from the upwards directed camera is processed into a feature image,
wherein a probability density function is calculated for the particles in the particle filter localization technique,
and wherein a most likely location is determined from the probability density function and set as the absolute position of the industrial vehicle.
Within the same field of endeavor as Brasile, Jeong teaches the following as described above in claim 1 with appropriate motivation:
wherein the image from the upwards directed camera is processed into a feature image, (Jeong Pg ¶ 0035 lines 1-4)
Within the same field of endeavor as Brasile and Jeong, Son teaches:
wherein a probability density function is calculated for the particles in the particle filter localization technique, and wherein a most likely location is determined from the probability density function and set as the absolute position of the industrial vehicle. (Son Pg 7 ¶ 3 “In the process of moving the particles, the spatial probability that the particles will be present is analyzed (S122), and as a result, when it is determined that particles are concentrated at a specific location on the digital map and particle location convergence is achieved (S124), the location is digital map of the positioning device. It is the gist of particle filter localization that is considered to be the actual location of the image (S126),” teaching setting the actual location (absolute position) by comparison (particle location convergence) of an expected feature image (digital map) for reach particle (particles are concentrated at a specific location on the digital map) analyzed using the spatial probability (probability density function))
Brasile, Jeong, Simon, and Son are both considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the floorplan map GPS navigation of Brasile and ceiling map navigation of Schmidt, and 3D digital environment feature map navigation of Simon by adding the spatial probability analysis and particle filter localization Son. This modification would be made with a reasonable expectation of success as motivated by efficient and low-cost computation (Son Pg 7 ¶ 1). While not relied upon for the rejections of Claims 2-4 as Son presents the solution with more explicit clarity, Jeong ¶ 0036 also describes localization through the use of particle probability.
Claim 5 is rejected under 35 U.S.C. 103 as being unpatentable over Brasile in view of Jeong and Simon and further in view of Zhou (CN 210402103, hereinafter “Zhou,” all citations and excerpts taken from the attached machine translation).
Regarding Claim 5, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 1 as described above. Brasile does not teach:
further comprising a step of calibrating the laterally-directed camera,
which step includes mapping at least one pixel on the images of said laterally-directed camera
to a distance
and direction relative to said industrial vehicle.
Within the same field of endeavor as Brasile, Zhou teaches:
further comprising a step of calibrating the laterally-directed camera, (Zhou Pg 5 ¶ 8 lines 8-9 “As shown in FIG. 4, the process of detecting and determining the obstacle depth camera is as follows:” teaching that the process is part of the procedure)
which step includes mapping at least one pixel on the images of said laterally-directed camera to a distance and direction relative to said industrial vehicle. (Zhou Pg 5 ¶ 8 lines 9-12 “the relative pose between the depth camera and the ground to be calibrated, wherein the pixel point according to the distance from the ground, normal vector, linked domain size constraint, determining to obtain obstacle); information obtained by the probe are contained in the obstacle information,” teaching the calibration of the pixel point in determination of depth and relative pose compared to the normal vector, which is equivalent to distance and direction)
Brasile and Zhou are both considered analogous because they both relate to camera-based obstacle identification. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the person detection of Brasile by adding the pixel point calibration procedure when determining depth and relative pose of an obstacle of Zhou. This modification would be made with a reasonable expectation of success as motivated by improving accuracy of measurement and preventing degradation of measurements over time.
Claim 13 is rejected under 35 U.S.C. 103 as being unpatentable over Brasile in view of Jeong and Simon and further in view of Nielsen (US 11835949, hereinafter “Nielsen”) and Kim et al (US 20220189058, hereinafter “Kim”)
Regarding Claim 13, the combination of Brasile, Schmidt, and Simon teaches the limitations of Claim 12 as described above. Brasile does not teach:
wherein the features comprise windows
and optionally other reflective surfaces,
and wherein the step of determining the absolute position of the industrial vehicle accounts for reflections of features in said windows
and optionally said other reflective surfaces.
Within the same field of endeavor as Brasile, Nielsen teaches:
wherein the features comprise windows and optionally other reflective surfaces, (Nielsen Col 6 line 64 – Col 7 line 11 “In some implementations, the control system 40 stores, in computer memory, a map of a space in which the robot travels. The map may be located on the robot or at any location remote from the robot that is accessible to the control system. The map includes locations of landmarks, such as […] windows, […] The map also may also include measurements indicating the size of the space, measurements indicating the size and locations of the landmarks, measurements indicating distances between landmarks, and coordinate information identifying where the landmarks are located in the space. The control system may use visual data from the vision system and data from the map to navigate throughout the space during operation,”)
Brasile, Simon, and Nielsen are both considered analogous because they both relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the floorplan map GPS navigation of Brasile and 3D digital environment feature map navigation of Simon by adding the mapped window features of Nielsen. This modification would be made with a reasonable expectation of success as motivated by applying a known technique to a known device (method or product) ready for improvement to yield predictable results (MPEP 2143(I)(D)).
The combination of Brasile and Nielsen does not teach:
and optionally other reflective surfaces, and wherein the step of determining the absolute position of the industrial vehicle accounts for reflections of features in said windows and optionally said other reflective surfaces.
Within the same field of endeavor as Brasile, Kim teaches:
and wherein the step of determining the absolute position of the industrial vehicle accounts for reflections of features in said windows and optionally said other reflective surfaces. (Kim ¶ 0058 “In FIG. 11(a), typical video analytics software might detect 1120 as a person, but 1120 is a reflected image of a person 1110 onto a standing mirror or a glass-type wall. The digital twin location management unit 230 knows a distance 1121 between the bottom end of 1120 and the surface level, which means 1120 is not a real person. On the other hand, 1110 can be confirmed as a real person—the bottom of center line 1111 meets with a point 1112 on the surface level,” applying to both person and feature recognition of Brasile, Simon, and Nielsen)
Brasile, Nielsen, and Kim are all considered analogous because they all relate to image recognition. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the floorplan map GPS navigation of Brasile, 3D digital environment feature map navigation of Simon, and mapped window features of Nielsen by applying the reflected image digital twin location management of Nielsen to the feature recognition of Brasile, Simon, and Nielsen. This modification would be made with a reasonable expectation of success as motivated by the use of known technique to improve similar devices (methods, or products) in the same way (MPEP 2143(I)(C)).
Claim 15 is rejected under 35 U.S.C. 103 as being unpatentable over Brasile in view of Jeong and Simon and further in view of Gyongyosi (WO 2018035482, hereinafter “Gyongyosi”).
Regarding Claim 15, the combination of Brasile, Jeong, and Simon teaches the limitations of Claim 14 as described above. Brasile does not teach:
wherein the three-dimensional feature map comprises racking intersections as features, wherein racking intersections are intersections of girders, beams or shelves of the racking and posts or uprights of the racking.
Within the same field of endeavor as Brasile, Gyongyosi teaches:
wherein the three-dimensional feature map comprises racking intersections as features, wherein racking intersections are intersections of girders, beams or shelves of the racking and posts or uprights of the racking. (Gyongyosi ¶ 0053 lines 1-3 “During the flight plan, side facing cameras on either side of the drone maintain the drone's position within the aisle by counting vertical uprights and horizontal cross-beams of the warehouse racks to determine its position relative to rack and level,” teaching the use of warehouse rack vertical uprights and horizontal cross-beams to locate an unmanned vehicle)
Brasile, Simon, and Gyongyosi are all considered analogous because they all relate to industrial vehicle navigation. Therefore, it would have been obvious to someone of ordinary skill in the art before the effective filing date of the claimed invention to have modified the floorplan map GPS navigation of Brasile and 3D digital environment feature map navigation of Simon by adding position determination based on camera detection of warehouse rack vertical uprights and horizontal cross-beams of Gyongyosi. This modification would be made with a reasonable expectation of success as motivated by the use of known technique to improve similar devices (methods, or products) in the same way (MPEP(I)(C)).
Conclusion
The prior art made of record and not relied upon is considered pertinent to applicant's disclosure.
US 9170581 and KR 101686170 apply generally to overhead-feature-based navigation
Any inquiry concerning this communication or earlier communications from the examiner should be directed to ZACHARY E GLADE whose telephone number is (703)756-1502. The examiner can normally be reached 4-5-9 7:30-16:30.
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Kito Robinson can be reached at (571) 270-3921. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/ZACHARY E. F. GLADE/Examiner, Art Unit 3664
/KITO R ROBINSON/Supervisory Patent Examiner, Art Unit 3664
Prosecution Insights