Detailed Action
1. Claims 1-9 are pending in this Application.
Notice of Pre-AIA or AIA Status
2. The present application, filed on or after March 16, 2013, is being examined under the first inventor to file provisions of the AIA .
Claim Rejections - 35 USC § 102
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the basis for the rejections under this section made in this Office action:
A person shall be entitled to a patent unless -
(a)(1) the claimed invention was patented, described in a printed publication, or in public use, on sale or otherwise available to the public before the effective filing date of the claimed invention.
3. Claims 1, 8 and 9 are rejected under 35 U.S.C. 102(a)(1) as being anticipated by HU et al., ( hereafter Hu), “Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection” IEEE, pub. 01/25/2021
As to claim 1, Hu teaches a method for detecting workpieces ( Abstract, page 1 right col. 2nd par., Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection. The proposed network is a lightweight generative architecture for grasping detection in one stage. A modified one- or two-stage deep learning object detection method, such as Faster R-CNN, to perform robotic grasping detection tasks using RGB-D image inputs) the method comprising;
performing data augmentation on an original image of an original workpiece (page 8, section B: data processing, Hu teaches a method using online data augmentation to train their network on most datasets, but they train directly on the Jacquard dataset without any data augmentation because it already has enough data).
performing training on a neural network model ( Fig.3, page 6 section D. Loss Function Hu teaches a method of training the lightweight generative grasping detection model to learn a grasp prediction function by minimizing the error between predicted grasps and true labels. It treats grasp pose estimation as a regression problem and uses the Smooth L1 loss function for detection) comprising multiple feature extraction branches to obtain a workpiece detection model based on a workpiece image group obtained through the data augmentation ( pages5-6, section B, Hu specifically teaches a multi-scale receptive field block (RFB) [34] to assemble the bottleneck layer of our grasping detection architecture for improving the ability of extracting multi-scale information and enhancing the feature discrimination.. The receptive field block module uses four branches with different convolution sizes to extract features, combines them, and adds them to the input data to create a multi-scale output);
converting the workpiece detection model into a lightweight workpiece detection model .Guassian-based grasping representation, we develop a lightweight generative architecture for robotic grasping pose estimation. Fig. 3 illustrates the structure of the lightweight generative grasping detection algorithm. I and Conv denote the input data and convolution filter, respectively. The proposed method consists of the downsampling block, the bottleneck layer, the multi-dimensional attention fusion network and the upsampling block shown in Fig.4)
performing detection on workpieces with the same shape and at least one different dimension as the original workpiece based on the lightweight workpiece detection model. (Fig. 8: page 7 left col., last two paragraphs, the detection results of grasping network on Cornell dataset. The first three rows are the maps for grasp quality, angle and width representing the opening and closing distance of the gripper. And, the last row is the best grasp outputs for several objects. Online data augmentation methods, including random cropping, zooms, and rotation, are applied to extend the dataset and avoid overfitting during training. The zooming and cropping operations changes the size of the object (workpiece) without altering the shape of the object).
Claim 8 is rejected the same as claim 1 except claim 8 is directed to an apparatus claim. The rejection of claim 1 includes all the limitations of claim 8. Thus, argument analogous to that presented above for claim 1 is applicable to claim 8.
As to claim 9, Hu teaches An electronic device comprising: a processor; and a memory storing an application program executable by the processor (Abstract page 8 section C Training Methodology, In training period, the generative model end to end trained on a Nvidia GTX2080Ti GPU with 22GB memory. The grasp detection algorithm achieves high accuracy and speed );
regarding the remaining limitations of claim 9, all the remaining limitations are the same as claim 1 except claim 9 is directed to apparatus claim. The rejection of claim 1 includes all the remaining limitations of claim 9. Thus, argument analogous to that presented above for claim 1 is applicable to remaining limitations of claim.
Claim Rejections - 35 USC § 103
In the event the determination of the status of the application as subject to AIA 35 U.S.C. 102 and 103 (or as subject to pre-AIA 35 U.S.C. 102 and 103) is incorrect, any correction of the statutory basis for the rejection will not be considered a new ground of rejection if the prior art relied upon, and the rationale supporting the rejection, would be the same under either status.
The following is a quotation of 35 U.S.C. 103 which forms the basis for all obviousness rejections set forth in this Office action:
A patent for a claimed invention may not be obtained, notwithstanding that the claimed invention is not identically disclosed as set forth in section 102, if the differences between the claimed invention and the prior art are such that the claimed invention as a whole would have been obvious before the effective filing date of the claimed invention to a person having ordinary skill in the art to which the claimed invention pertains. Patentability shall not be negated by the manner in which the invention was made.
4. Claim 3 is rejected under 35 U.S.C. 103 as being unpatentable Hu, “Lightweight Convolutional Neural Network with Gaussian-based Grasping Representation for Robotic Grasping Detection, in view of Wei Chen et al., (hereafter Wei) “ FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose “Estimation with Decoupled Rotation Mechanism’, IEEE, pub.2021 (IDS Item)
Regirding Claim 2, while HU teaches the limitation of Calim1, but fails to teach the limitation of claim 2.
On the other hand in the same field of endeavor machine learning based image processing of Wei teaches performing data augmentation on an original image of an original workpiece comprises: generating a new workpiece image in the workpiece image group based on the original image, a workpiece in the new workpiece image has the same shape as the original workpiece; wherein the length of workpiece in the new workpiece image is the same as that of the original workpiece and the width of workpiece in the new workpiece image is different from that of the original (workpiece); or wherein the width of workpiece in the new workpiece image is the same as that of the original workpiece and the length of workpiece in the new workpiece image is different from that of the original workpiece ( Fig. 5, section 3.5. 3D Deformation Mechanism, To fix shape variation, Wei specifical teaches an online box-cage 3D deformation method for data augmentation. This uses pre-defined cages for rigid objects where points move along with surface deformations. Figure 5. 3D deformed examples: The new training examples can be generated by enlarging, shrinking, or changing the area of some surfaces of the box-cages. The left one is the original point could with original 3D box-cage, i.e. 3D bounding box. The right three ones are the deformed point clouds with deformed box cages(shown in yellow color). The green boxes are the original 3D bounding boxes before deformation..
It would have been obvious to a person of ordinary skill in the art, before the effective filing date of the claimed invention, to incorporate the online box-cage 3D deformation method into Hu’s grasp detection framework. The motivation for doing so would have been to allow users of Hu to advance data augmentation from 2D pixel-level variations to robust 3D geometric shape variations. This shift helps the network handle unseen object dimensions and complex structures.
5. Claims 3-7 are objected to as being dependent upon a rejected base claims, but would be allowable if rewritten in independent form including all of the limitations of the base claim and any intervening claims.
6. Regarding dependent claim 3 no prior art is found to anticipate or render the flowing limitation obvious;
“wherein generating a new workpiece image in the workpiece image group based on the original image comprises: dividing the original workpiece in the original image into a first region with invariant features and a second region with variable features; changing the length of the second region to generate a second region with changed length; changing the width of the second region to generate a second region with changed width; combining the first region and the second region with changed length to form the new workpiece image; or combining the first region and the second region with changed width to form the new workpiece image.
7. Claims 6 and 7 are objected to because they are dependent of the objected claim dependent claim 3.
8. Regarding dependent claim 4 no prior art is found to anticipate or render the flowing limitation obvious:
“obtaining a meta transformer; replacing a feature extraction layer of the meta transformer with multiple feature extraction branches, wherein the multiple feature extraction branches comprise at least one of the following: shortcut to adder, 1x1 convolutional kernel in parallel with pooling unit, and 1x1 convolutional kernel in parallel with convolutional unit; determining the replaced meta transformer as the neural network model.”
9. Claim 5 is objected to because it depends on the objected claim 4.
Prior arts are not used in rejections but pertinent to the claims or disclosure.
a.. “LeanNet: An Efficient Convolutional Neural Network for Digital Number Recognition in Industrial Products”, Sensors 2021, by Na Qin et al., disclosed:
The remarkable success of convolutional neural networks (CNNs) in computer vision tasks is shown in large-scale datasets and high-performance computing platforms. However, it is infeasible to deploy large CNNs on resource constrained platforms, such as embedded devices, on account of the huge overhead. To recognize the label numbers of industrial black material product and deploy deep CNNs in real-world applications, this research uses an efficient method to simultaneously (a) reduce the network model size and (b) lower the amount of calculation without compromising accuracy. More specifically, the method is implemented by pruning channels and corresponding filters that are identified as having a trivial effect on the output accuracy. In this paper, we prune VGG-16 to obtain a compact network called LeanNet, which gives a 25× reduction in model size and a 4.5× reduction in float point operations (FLOPs), while the accuracy on our dataset is close to the original accuracy by retraining the network. Besides, we also find that LeanNet could achieve better performance on reductions in model size and computation compared to some lightweight networks like MobileNet and SqueezeNet, which are widely used in engineering applications. This research has good application value in the field of industrial production (see Abstract)
b. “Fusing few-shot learning and lightweight deep network method for detecting workpiece pose based on monocular vision systems” Measurement 218 (2023) 113118, ELSEVIER, to T. Zhang et al., disclosed;
In workpieces pose measurement of industrial grasping or sorting, aiming at the problems of slow measurement speed, low accuracy, high resource cost of measurement data processing, and poor ability of measurement algorithm to adapt to the new workpiece, a lightweight workpiece pose detection algorithm with few-shot learning is proposed in the paper. The full paper is summarized as follows: (1) To realize the pose measurement of the workpiece, angle pa rameters and rotation loss function are introduced based on YOLOv4-Tiny to change the original horizontal detection box into a rotating detection box. Then, the lightweight RLOYO-MNV3 network is built by integrating the lightweight module of Mobi leNetV3 to improve the measurement speed. Finally, a two-stage fine-tuning strategy is combined to enhance the adaptability of the algorithm to new workpieces. (2) To verify the performance of the algorithm, the proposed algo rithm is trained and tested by building datasets of 10 different workpieces. The test results show that the parameter number of the proposed method is 1.11 M, which is reduced by 81.7% compared with the baseline RYOLO-Tiny, and the required FLOPs is 0.632G, which is increased by 5.49 times, while the average prediction accuracy is only reduced by 1.97%. And when the sample number is 50, the mAP of the two new class partitioning methods is close to 83%. Therefore, under the condition of ensuring measurement accuracy, the proposed algorithm greatly reduces the resource cost and improves the measurement speed, and has good adaptability to the new workpiece ( see section 6 page 12, 6. Conclusion and discussion).
Contact Information
Any inquiry concerning this communication or earlier communication from the examiner should be directed to Mekonen Bekele whose telephone number is (469) 295-9077.The examiner can normally be reached on Monday -Friday from 9:00AM to 6:50 PM Eastern Time.
If attempt to reach the examiner by telephone are unsuccessful, the examiner’s supervisor Eng, George can be reached on (571) 272-7495.The fax phone number for the organization where the application or proceeding is assigned is 571-237-8300. Information regarding the status of an application may be obtained from the patent Application Information Retrieval (PAIR) system. Status information for published application may be obtained from either Private PAIR or Public PAIR.
Status information for unpublished application is available through Privet PAIR only.
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should you have question on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866.217-919 (tool-free)
/MEKONEN T BEKELE/Primary Examiner, Art Unit 2699
Prosecution Insights