Notice of Pre-AIA or AIA Status
The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
112(f) Invocation
The following is a quotation of 35 U.S.C. 112(f):
(f) Element in Claim for a Combination. – An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
The following is a quotation of pre-AIA 35 U.S.C. 112, sixth paragraph:
An element in a claim for a combination may be expressed as a means or step for performing a specified function without the recital of structure, material, or acts in support thereof, and such claim shall be construed to cover the corresponding structure, material, or acts described in the specification and equivalents thereof.
This application includes one or more claim limitations that do not use the word “means,” but are nonetheless being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, because the claim limitation(s) uses a generic placeholder that is coupled with functional language without reciting sufficient structure to perform the recited function and the generic placeholder is not preceded by a structural modifier. Such claim limitation(s) is/are:
image processing module, link system, module, navigation module; guidance module; extraction module; on board module; down sampling; residual module; mode setting module; protection module; data dumping module and monitoring module
in claims 1-5, 7, 8 and 10-14.
Because this/these claim limitation(s) is/are being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, it/they is/are being interpreted to cover the corresponding structure described in the specification as performing the claimed function, and equivalents thereof.
If applicant does not intend to have this/these limitation(s) interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph, applicant may: (1) amend the claim limitation(s) to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph (e.g., by reciting sufficient structure to perform the claimed function); or (2) present a sufficient showing that the claim limitation(s) recite(s) sufficient structure to perform the claimed function so as to avoid it/them being interpreted under 35 U.S.C. 112(f) or pre-AIA 35 U.S.C. 112, sixth paragraph.
Claim Rejections - 35 USC § 112
The following is a quotation of the first paragraph of 35 U.S.C. 112(a):
(a) IN GENERAL.—The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor or joint inventor of carrying out the invention.
The following is a quotation of the first paragraph of pre-AIA 35 U.S.C. 112:
The specification shall contain a written description of the invention, and of the manner and process of making and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to make and use the same, and shall set forth the best mode contemplated by the inventor of carrying out his invention.
Claims 1-17 are rejected under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), first paragraph, because the specification, while being enabling for Euclidean algorithm, does not reasonably provide enablement for this algorithm as the claimed “integration algorithm”. The specification does not enable any person skilled in the art to which it pertains, or with which it is most nearly connected, to associated the integration algorithm as the Euclidean algorithm set forth in para. 76 of the specification commensurate in scope with these claims. Correction is required.
Claim Rejections - 35 USC § 103
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
Claim(s) 1, 5 and 12-14 is/are rejected under 35 U.S.C. 103 as being unpatentable over CN116540564 in view of Kaminka (20180311822) further in view of KSR v. Teleflex 550 U.S. 398 (2007).
With respect to claim 1, the 564 reference teaches an intelligent inspection system for detecting crack information, see the Background (2nd Full Para. beginning at line 4) regarding track detection along a railway line, comprising:
a track detection robot (patrol trolley) which has three linear cameras fixed on the front of the trolley. See the Background at para. 2.
The 564 reference further teaches the unmanned vehicle (inspection trolley) that acquires image information from the plurality of cameras. Background, at para. 2, beginning at line 3. The image information is acquired at steps S4 and S5, see page 3, where the information are cracks associated with the track plate.
The 564 reference teaches a remote server configured to store and received signals form the patrol trolley. See the Background at Para. 2, lines 5-6.
The 564 reference teaches a computer, described at the Background, 2nd Para. lines 12-13 for processing a detection model, described in the 5th para. for analyzing and computing the results of the image data obtained by cameras.
The 564 reference teaches a processing module (detection model) described at para. 2, using deep learning networks (see the page 4, lines 1-5). The 564 reference detects rail defects (see Background para. 2, line 7. The 564 reference teaches
Displaying the defect results at step S5 and the second to last para. at page 3. Based on the report and display by the detection model, the defects may be identified and corrected by the appropriate personnel.
The 564 reference teaches preprocessing at page 8, line 6 as claimed.
The 564 reference teaches histogram equalization, which best as can be determined, is similar to the Euclidean algorithm mentioned in para. 13 of the instant specification.
The 564 reference teaches a deep learning network for faster detection for defects on railroads using CUDA 11.3 and CUDNN 8.2 to improve model reading speed as stated at page 6, lines 7 and 8 the image processing module.
The 564 reference teaches converting the image information to BGR format through Cv_bridge beginning at line 11 and at section(2) at page 6.
The 564 reference teaches YOLO v5 (see page 6, line 5), for detecting defects which may include bridges upon which the defective tracks are laid.
What is not specifically taught by the 564 reference is that the trolley is a UAV.
The 564 reference teaches a rod fixedly placed in front of the trolley for holding the linear arrayed sensors, see The Background, 2nd para., lines 3-4.
While the trolley is autonomous, it is not airborne. It does obtain aerial images by means of a linear array disposed on a rod connected to the trolley.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to substitute ariel photos from a trolley by those of a drone, as a matter of convenience, for obtaining defects in the rail system.
The 564 reference fails to teach a mobile ground base station as claimed and determining the coordinates of defects.
Kaminka teaches at least one UAV that is charged by a mobile ground station as taught at para. 53.
Kaminka is also directed to detecting cracks or defects in a panel. See para. 64
Kaminka also teaches a control station that determines the coordinates of all robots for service, see para. 139. This includes those that inspect surfaces of cracks taught at para. 64.
Since the 564 reference and Kaminka are directed to autonomous vehicles that determined imperfections by use of the autonomous vehicle, the purpose of using an aerial device having a mobile ground station and a control of where defects have been identified would have been recognized by the 564 reference as set forth by Kaminka.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teaching of Kaminka with the 564 reference to use a movable base station, a control for identifying the location of defects for the purpose of charging autonomous vehicles and identifying the location of defects as set forth by Kaminka.
The 564 reference does not specifically teach the use of CYOLO. However, the 564 reference teaches CUDA 11.3
and CUDNN 8.2 which appears as substitutes or near equivalents for improving the speed of detecting defects, as stated in the 2nd full paragraph of page 6.
Hence, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the clamed invention, to try different algorithms suitable for rendering faster, yet reliable data points for identifying defects found in images pertaining to a railway.
The 564 reference does not make specific reference to RGBNet. The 564 reference makes teaches BGR conversion through Cv_bridge, see page 6, lines 11 and 28. The BGR conversion appers to be a This appears to be a near equivalent or would render similar results as the RGBNet, for at least the reason that the RGBNet and converted BGR carrying out enhancements on images and highlighting or emphasizing important features such as rail boundaries as claimed.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to try one conversion algorithm in place of another for post processing images for enhancing them suitable for identifying defects of objects detected on a railway system.
The 564 reference does not specifically make reference to an abnormality algorithm, however, the 564 reference teaches performing image smoothing, which appears to reduce abnormalities that would otherwise distort images.
Hence, it would have been obvious to one of ordinary skill in the art before the effective filing data of the claimed invention, to try a smoothing algorithm as an abnormality algorithm for the purpose of enhancing processed images based on multi-sourced data (data taken from three linear arrays - see Background at para. 2, line 3), fused or integrated and processed in accordance with a railroad environments (over tracks and bridges).See also the Abstract at line 6.
With respect to claim 5, the 564 reference teaches YOLOv5 (see page 6, line 6). The reference is further directed to finding defects in a railway line which includes steel structures as the bridge is part of the railway system.
Furthermore, the 564 reference teaches a detection algorithm based on CNN, see the second full para. at line 8.
Hence, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, that the YOLOv5 taught within the 564 reference, can be used for imaging defects in steel structure of a bridge for at least the reason that the same YOLOv5 process is being used in both the prior art and the present invention. Hence, the YOLOv5 would also be applicable in steel structures though not specifically mentioned by the 564 reference.
With respect to claim 12, the 564 reference does not teach the ground mobile base. However, Kaminka was relied upon for the teaching of the ground mobile base. As previously stated, Kaminka teaches at least one UAV that is charged by a mobile ground station as taught at para. 53.
Kaminka is also directed to detecting cracks or defects in a panel. See para. 64
Kaminka also teaches a control station that determines the coordinates of all robots for service, see para. 139. This includes those that inspect surfaces of cracks taught at para. 64.
Kaminka teaches one or more base stations that serve as charging stations (see para. 97). Kaminka also teaches that the base stations are where the robots are serviced, recharged and resupplied, see (para. 100). Hence the servicing and resupplying as referred to by Kaminka includes the servicing and resupplying of charging devices (batteries). Kaminka teaches transport robots (paras. 94 and 98) may perform automatic functions ( para. 109 ) which include controlling transporting and delivery (loading) of service robots (para. 99).
Kaminka also teaches automatic communication between service robots and transport robots and vice versa, see para. 50. Kaminka teaches wherein the control stations serve as a module for dumping data (transmitting data from different sources), see para. 147. At the bottom of para. 144, Kaminka teaches “… multiple robots may collect or deposit other robots into base stations. At para. 105, base stations perform computational processes to control other robots which includes communication. Therefore, it would have been obvious to one of ordinary skill in the art before the effective filing data of the claimed invention, that acquired data is that which can be communicated to other robots/ground base stations.
Kaminka also teaches a plurality of sensor (see para. 52) that have emitter elements, and carrier elements that allow the service and transport robots to engage. See also para. 53. The sensor, emitter and carrier elements function as a module for allowing a UAV service robot to be released by a UAV transport robot. Therefore, it is a digital communication that automatically causes the transport robot to release the service robot, see para. 51.
Kaminka teaches a take off operation (with regard to pick up of an object, see para. 51, line 10) and the landing of the UAV (with regarding to “transport/drop-off”, see para. 51, line 11). With regard to the one button operation, at para. 75, Kaminka teaches the operation can be humanly generated. So a human could push a button to generate the communication which would facilitate the landing operations as described in the claim and by para. 51 of Kaminka.
Regarding the limitation of energy support for field operations, this limitation is contemplated by the battery power that is used to provide energy from the transport robot to the service robot wherein the service robot provides support for a field operation, such as cleaning a surface. Para. 51, lines 8-13 and para. 68.
Since the 564 reference and Kaminka are directed to autonomous vehicles that determined imperfections by use of the autonomous vehicle, the purpose of using an aerial device having a mobile ground station and a control of where defects have been identified would have been recognized by the 564 reference as set forth by Kaminka.
It would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to combine the teaching of Kaminka with the 564 reference to use a movable base station, a control for identifying the location of defects for the purpose of charging autonomous vehicles and identifying the location of defects as set forth by Kaminka.
Any additional motivation for the rejection to claim 12 is provided in the rejection to claim 1.
With respect to claim 13, Kaminka teaches a system that has applicability on railway systems because it uses an aerial autonomous vehicle for inspecting defects on a surface, see para. 76.
Kaminka teaches software instructions, (taught by paras. 283 and 289) for performing the method steps of the monitoring module wherein, autonomous or automatic preconfigured instructions are used to control the robots to perform the functions, see para. 73, lines 1-6. This means that the operating path of the UAV, or its flight instructions, have been preconfigured. Moreover, para. 73, lines 1-6 suggests that its flight path or method of operation can be modified.
At para. 131, lines 1-5 and 8-15, Kaminka teaches that the use of passive or active sensing devices for one or more sensing markers, may be used. At para. 133, the sensors and markers are used to control the robots.
Hence, the flight plans are preconfigured instructions which can be modified in order to determine the flight route. The flight can be monitored by a human. Paras. 105, 108 suggests that the fight path or mission can be changed in real-time as the flight information for a robot is displayed on a visual screen (see para. 140).
Kaminka does not specifically state that the fight data is generated by use of LIDAR. However, the limitation is suggested by Kaminka. For example, at para. 154, Kaminka teaches that surfaces can be sensed by different modalities. Kaminka further distinguishes the use of visible and invisible light including their reflections from the surface. The Examiner contends that LIDAR uses reflected light in its detection. Kaminka is using reflected light in its detection of defects on an object surface.
Hence, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use LIDAR as a measurement for detecting defects on a surface as clearly taught by Kaminka.
.
Kaminka teaches data dumping modules via control station, identified at para. 138, lines 7-9 and 139, lines 1-5
The Kaminka reference teaches that the control station receives data from other robots and sources (mobile base station at para. 103) and keeps them for historical analysis not just in real-time. The Examiner contends that it is a matter or common knowledge, or would have been obvious to one of ordinary skill in the art before the effective filing date of the claimed invention, that the control station must use a storage device if historical information is to be used futuristically and not held in real-time (para. 139, lines 4, 5).
As to the control station performing a classification of the information, clearly, Kaminka teaches performing various computational processes (para. 138, lines 6-7) which include the determination of defects on surfaces see para. 138, line 8 – para. 139, line 2. Furthermore, after analyzing the results, they are shared with other operators (para. 138, last line). Hence, other operators would have been understood to be those at other facilities along the railway lines.
As to the storage of data to an equipment database from the dumping module, the Examiner contends that the dumping module is the control station. Kaminka teaches the control station relaying processing results and other information about defects to other control stations (equipment with storage/database – see para. 138, last line, para. 140, lines 1 and 2 and para. 141 line 5.
The motivation for the rejection to claim 13 is the same as that to claim 1 upon which this claim depends.
With respect to claim 14, Kaminka teaches geographic coordinates for all flight patterns including, hoovering maneuvers, cruising and all other flight control procedures. See para. 213 regarding GPS as geographic coordinates. See also para. 73, lines 1-8 for autonomous and automatic flight control processes.
Regarding 3-dimensional modeling of the flight trajectory, see para. 175. See para. 154 regarding taking of images in the target area.
What is not specifically taught is point cloud data generated by LIDAR.
However, the limitation is suggested by Kaminka. For example, at para. 154, Kaminka teaches that surfaces can be sensed by different modalities. Kaminka further distinguishes the use of visible and invisible light including their reflections from the surface. The Examiner contends that LIDAR uses reflected light in its detection. Kaminka is using reflected light in its detection of defects on an object surface.
Hence, it would have been obvious to one of ordinary skill in the art, before the effective filing date of the claimed invention, to use LIDAR as a measurement for detecting defects on a surface as clearly taught by Kaminka.
Any additional motivation for the rejection to claim 14 is provided in the rejection to claim 13.
Claims Objected As Containing Allowable Subject Matter
Claim 2-4, 6-11, 15-17 would be allowable if rewritten or amended to overcome the rejection(s) under 35 U.S.C. 112(a) or 35 U.S.C. 112 (pre-AIA ), 1st paragraph, set forth in this Office action.
Claim 2, if corrected, would be allowed for the reason the prior art does not teach the claimed combination of “the infrared thermal imaging module….presents the operating state of the railway infrastructure and equipment by means of a temperature measurement function.”
Claim 7 depends from claim 2 and contains allowable subject matter for similar reasons.
Claim 8 depends from claim 7 and contains allowable subject matter for similar reasons.
Claim 9 depends from claim 7 and contains allowable subject matter for similar reasons.
Claim 10 depends from claim 2 and contains allowable subject matter for similar reasons.
Claim 11 depends from claim 2 and contains allowable subject matter for similar reasons.
Claim 3, if corrected, would be allowed for the reason the prior art does not teach the claimed combination, wherein the RBGnet-based rail surface segmentation algorithm consists of a supervised saliency model, a backbone network module, an extraction module for rail and surface saliency features.
Claim 4 depends from claim 3 and contains allowable subject matter for similar reasons.
Claim 6, if corrected would be allowed for the reason the prior art does not teach the abnormality detection algorithm based on segmenting the point cloud data by using a large-scale point cloud semantic segmentation model based on random sampling, feature aggregation and prototype fitting, clustering the point cloud by using an improved Euclidean algorithm.
Claim 15, if corrected would be allowed for the reason the prior art does not teach detecting and identifying the defects in the newly received image information via the deep learning network model in the image processing module, sending a defect warning result for manual review, uploading the defect and specific location information after the defect is manually confirmed.
Claim 16 depends from claim 15 and contains allowable subject matter for similar reasons.
Claim 17 depends from claim 16 and contains allowable subject matter of similar reasons.
Any inquiry concerning this communication or earlier communications from the examiner should be directed to JEROME GRANT II whose telephone number is (571)272-7463. The examiner can normally be reached M-F 9:00 a.m. - 5:00 p.m..
Examiner interviews are available via telephone, in-person, and video conferencing using a USPTO supplied web-based collaboration tool. To schedule an interview, applicant is encouraged to use the USPTO Automated Interview Request (AIR) at http://www.uspto.gov/interviewpractice.
If attempts to reach the examiner by telephone are unsuccessful, the examiner’s supervisor, Jennifer Mehmood can be reached at 571-272-2976. The fax phone number for the organization where this application or proceeding is assigned is 571-273-8300.
Information regarding the status of published or unpublished applications may be obtained from Patent Center. Unpublished application information in Patent Center is available to registered users. To file and manage patent submissions in Patent Center, visit: https://patentcenter.uspto.gov. Visit https://www.uspto.gov/patents/apply/patent-center for more information about Patent Center and https://www.uspto.gov/patents/docx for information about filing in DOCX format. For additional questions, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000.
/JEROME GRANT II/Primary Examiner, Art Unit 2664